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Overview 


Children’s school readiness as they enter kindergarten is an important predictor of their academic 
success and ultimately their long-term health and economic outcomes. Research shows that participa- 
tion in high-quality early childhood education helps preschoolers gain the foundational language and 
literacy skills as well as the social-emotional competencies they need to be ready for kindergarten. 
Even so, free or affordable high-quality preschool options remain limited for families in many under- 
served communities, and children growing up in poverty often have less access to the kinds of early 
learning opportunities that contribute to school readiness. 


Jumpstart is a nonprofit organization that aims to address the school readiness gap. It partners with 
early childhood education centers across the country to increase their capacity to provide high-quality 
language and literacy instruction to children in underserved communities. Jumpstart’s program model, 
delivered by volunteers, includes weekly curriculum-based sessions focused on children’s language 
and literacy development, as well as increased opportunities for volunteers to interact with children 
one-on-one and contribute to their language and social-emotional development in a less-structured 
format called child-centered time (CCT). 


With support from the Corporation for National and Community Service (CNCS), MDRC evaluated 
Jumpstart’s Foster Grandparent Program (JFG) in Los Angeles and Compton, California, where 
Jumpstart’s services were delivered by older adults from the community (Community Corps members 
or “Foster Grandparents’’). The evaluation examines whether the JFG program, which operated from 
fall 2011 to summer 2019, was implemented as intended and in what ways the volunteers enhanced 
children’s educational and developmental experience in the classroom. Using a child-level, random 
assignment research design, this evaluation also explores whether the JFG program model shows 
promise for improving children’s language and literacy and social-emotional development. 


Key Findings 
This evaluation found that: 


e The Foster Grandparents had consistently high attendance at training sessions and reported feeling 
adequately prepared to provide services to children. 


e The Foster Grandparents demonstrated a strong commitment to the program, and children expe- 
rienced an increased number of adults in the classroom. 


e The Foster Grandparents implemented the curriculum-based sessions regularly and as intended. 
While implementing the sessions, volunteers used various instructional practices aimed at im- 
proving children’s language and literacy development, but the frequency with which these prac- 
tices were used in the classroom varied across centers. 


e The Foster Grandparents devoted a high number of hours to CCT. However, it is not possible 
to determine how much of that time was spent one-on-one with partner children versus providing 
general classroom support. 


e Children who received services from volunteers made gains in language and literacy develop- 
ment, but the gains were not greater than those made by children who did not receive services 
from volunteers. 
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Executive Summary 


Children’s school readiness as they enter kindergarten is an important predictor of their academic 
success and ultimately their long-term health and economic outcomes.! Research shows that par- 
ticipation in high-quality early childhood education helps preschoolers gain the foundational lan- 
guage and literacy skills as well as the social-emotional competencies they need to be ready for 
kindergarten.” Even so, free or affordable high-quality preschool options remain limited for fam- 
ilies in many underserved communities, and children growing up in poverty often have less access 
to the kinds of early learning opportunities that contribute to school readiness.* Gaps in language 
and literacy skills begin early and widen over time.* 


Jumpstart is a nonprofit organization that aims to address the school readiness gap. It 
partners with early childhood education centers across the country to increase their capacity to 
provide high-quality language and literacy instruction to children in underserved communities.° 
Jumpstart’s program model, delivered by volunteers, includes weekly curriculum-based sessions 
focused on children’s language and literacy development (“Jumpstart sessions”’), as well as one- 
on-one time outside of those sessions (“child-centered time” or CCT). Most of Jumpstart’s vol- 
unteers are college-age students (“College Corps’’), but some volunteers are older adults from the 
community (Community Corps members or “Foster Grandparents”).° 


This report presents the findings from an evaluation of Jumpstart’s Foster Grandparent 
Program (JFG) in Los Angeles and Compton, California, which operated from fall 2011 to sum- 
mer 2019, where Jumpstart’s services were delivered by older adults. The evaluation, which fo- 
cuses on the 2017-2018 academic year, examines whether the JFG program was implemented as 
intended and to what extent the presence of volunteers enhanced children’s educational and de- 
velopmental experience in the classroom. The evaluation also explores whether the JFG program 


‘Christopher Blodgett and Myah Houghten, Every Child School Ready: Community, School, and Student 
Predictors of Kindergarten Readiness and Academic Progress (Olympia, Washington: Education Research & 
Data Center, 2018); Amy Pace, Rufan Luo, Kathy Hirsh-Pasek, and Roberta Golinkoff, “Identifying Pathways 
Between Socioeconomic Status and Language Development,” Annual Review of Linguistics 3 (2017): 285-308. 

?Amie Bettencourt, Deborah Gross, and Grace Ho, The Costly Consequences of Not Being Socially and 
Behaviorally Ready by Kindergarten: Associations with Grade Retention, Receipt of Academic Support Services, 
and Suspensions/Expulsions (Baltimore: Baltimore Education Research Consortium, 2016); Carmen Shery 
Brown, “Language and Literacy Development in the Early Years: Foundational Skills that Support Emergent 
Readers,” Language and Literacy Spectrum 24 (2014): 35-49; Pre-Kindergarten Task Force, The Current State 
of Scientific Knowledge on Pre-Kindergarten Effects (Washington, DC: Brookings Institute, 2017). 

3U.S. Department of Education, A Matter of Equity: Preschool in America (Washington, DC: U.S. Depart- 
ment of Education, 2015). 

4Anne Fernald, Virginia A. Marchman, and Adriana Weisleder, “SES Differences in Language Processing 
Skill and Vocabulary are Evident at 18 Months.” Developmental Science 16, 2 (2013): 234-248. 

5Jumpstart partners with higher education institutions, community organizations, Head Start programs, com- 
munity-based preschools, and school districts. 

® According to Jumpstart yearly reports, 4,054 Jumpstart volunteers served a total of 13,035 preschool chil- 
dren across the country in 2017-2018 (Jumpstart, National Evaluation of Jumpstart: 2017-2018 Program Year 
[Boston: Jumpstart, 2019]). 


model shows promise for improving children’s language and literacy and social-emotional devel- 
opment, which includes children’s expression and management of emotions, as well as the ability 
to establish positive and rewarding relationships with others (that is, interpersonal skills). The 
Corporation for National and Community Service (CNCS) funded the evaluation. 


Jumpstart Foster Grandparent Program Overview 


Jumpstart is a national organization that works with early childhood education centers across the 
country. Jumpstart’s national office is in Boston and the organization’s national staff provides 
support and broad oversight related to curriculum design, volunteer recruitment, and evaluation 
and monitoring. Jumpstart also has offices nationwide (including in Los Angeles), where staff 
members support the implementation of the programs. This includes site managers who are re- 
sponsible for overseeing the implementation of the model in several preschool classrooms at one 
or more centers. 


Jumpstart offered Foster Grandparents several types of pre-service and in-service training 
to prepare and support them as they delivered the Jumpstart model. Volunteers were expected to 
participate in eight weeks of pre-service training (three days a week, six hours a day, totaling 
more than 138 hours). The pre-service training included an orientation about the Jumpstart cur- 
riculum, early childhood education, classroom management, and other relevant topics. The vol- 
unteers were also expected to attend monthly in-service trainings (four hours each), which were 
facilitated by site managers. 


Each early childhood education center was assigned a team of four to seven Foster Grand- 
parents who supported it for the entire school year.” Each volunteer team was overseen and sup- 
ported by the Jumpstart site manager for that center, who observed and provided regular feedback 
to the volunteers. On days when volunteers were on-site, they were expected to meet after lunch 
for 1 to 1.5 hours to prepare and practice their delivery of the Jumpstart session plan for that week. 
On Fridays, the site managers also led a Team Leadership Meeting at a central location, where 
they reviewed and demonstrated the following week’s session plan to the volunteer teams. 


The overall goal of the JFG program was to place more trained adults in early childhood 
education classrooms so the children could receive more individualized attention, with a focus on 
their language, literacy, and social-emotional development. The Foster Grandparents provided 
this attention in two ways: Jumpstart sessions and child-centered time (CCT): 


e Jumpstart sessions are based on a language and literacy curriculum designed 
to supplement existing preschool curricula.* The sessions are intended to cre- 
ate opportunities for the volunteers to interact with children in a more struc- 
tured format. At the time of the study, the Jumpstart sessions targeted the de- 
velopment of three areas of foundational language and literacy skills in 


7The number of volunteers in each center was based on the goal of having one volunteer for every three 
children during the Jumpstart session. 


8Jumpstart’s curriculum was adapted from the Opening the World of Learning (OWL, v. 2005) curriculum. 


children: oral language (vocabulary, comprehension); book and print aware- 
ness (alphabet knowledge and use of print); and phonological awareness (pho- 
nemic and rhyme). The content of the curriculum, as well as the supportive 
interactions between children and the Foster Grandparents as they engaged in 
curriculum-based activities, were also intended to support children’s social- 
emotional development. As part of the JFG program, the Jumpstart sessions 
were offered in one designated classroom in each early childhood education 
center. (This classroom will be referred to as the “Jumpstart classroom.) Two 
mornings a week, the volunteers led a structured two-hour session with the 
children in that classroom. The Jumpstart sessions took place during regularly 
scheduled classroom time and were led by the volunteers. 


e While Jumpstart sessions were intended to provide a structured format for vol- 
unteer-child interactions, the second component of the JFG model — child- 
centered time (CCT) — consisted of time outside of the formal sessions, 
where volunteers interacted with children one-on-one in a less-structured for- 
mat. As an integral part of service, CCT gave volunteers opportunities to in- 
teract with their partner children and contribute to their language and social- 
emotional development within the setting of teacher-led instructional time. 
Foster Grandparents in the JFG program played important roles in the class- 
room by building strong relationships with children; by increasing the number 
of adults and thereby increasing opportunities for children to receive individ- 
ualized attention (from both volunteers and teaching staff); and by supporting 
children’s participation and learning during activities planned by classroom 
teachers. Foster Grandparents were also expected to build collaborative rela- 
tionships with teachers and supported teachers in the classroom. 


The implementation of the JFG program components (and the resulting increase in the 
adult-to-child ratio) was intended to increase the amount of one-on-one attention that children 
received and, in turn, to lead to improvements in their language and literacy development and 
their social-emotional development. 


Overview of the Evaluation 


The purpose of this evaluation is to expand the body of research on the Jumpstart model, focusing 
on how the model was delivered by Foster Grandparents. Many interventions rely on volunteers 
to provide services to children and youth; this study also aims to build broader knowledge about 
the implementation and potential effects of educational interventions that mobilize volunteers, 
especially programs like JFG, which required a relatively large time commitment. The findings 
from this study can also inform policy and practice about the potential benefits of intergenera- 
tional programs. 


The evaluation of Jumpstart includes an implementation study and an impact study. The 
goal of the implementation study is to describe the content, quantity, quality, and structure of 


services that the Foster Grandparents provided to children in the study centers. The goal of the 
impact study is to explore whether the full Jumpstart model and its components, as delivered by 
the Foster Grandparents, show promise for improving children’s outcomes. Eleven early child- 
hood centers in Los Angeles were included in the implementation study. The effect of the 
Jumpstart model was examined using a child-level random assignment research design in five of 
the eleven centers that were a part of the implementation study. In each of the five centers, chil- 
dren were randomly assigned to one of three types of classroom: (1) a Jumpstart classroom that 
received the full JFG model, and where all children received the Jumpstart sessions and child- 
centered time outside of the Jumpstart sessions; (2) a CCT-only classroom, where children did 
not receive the Jumpstart sessions but where some children received child-centered time; or (3) a 
“business as usual” classroom that did not receive services from the volunteers. The impact of 
the Jumpstart model can be estimated by comparing the spring outcomes of children in the class- 
rooms served by the volunteers to the spring outcomes of children in the business as usual class- 
rooms. 


The implementation and impact studies in this evaluation are based on several data 
sources. For the impact study, children’s outcomes were measured using a teacher-reported as- 
sessment called the Desired Results Developmental Profile — Preschool. The DRDP-PS is an ob- 
servational tool that can be used to rate children’s development in several domains. Early child- 
hood education centers in California that receive state funding (including all 11 study centers) 
must use the DRDP-PS to assess each child’s development in the fall and spring. For the imple- 
mentation study, the study team used data collected by Jumpstart to internally monitor the pro- 
gram, including data on child attendance during the Jumpstart session; training attendance records 
and volunteer timesheets; a pre-service and a post-service volunteer survey; and an observation 
checklist used by Jumpstart site managers when they observed volunteers in the Jumpstart ses- 
sions. The study team also conducted interviews with volunteers and administered a survey to all 
lead teachers in the study centers. Because teachers were present while volunteers worked with 
children, the survey asked teachers to report on volunteers’ use of different instructional practices 
aimed at promoting children’s language and literacy development, as well as their own use of 
these strategies.° 


Implementation and Service Contrast 


To better understand the context in which the JFG program was executed, the implementation 
study examined the content, quantity, quality, and structure of services that the Foster Grandpar- 
ents provided to children in the study centers. It also examined the extent to which the presence 
of the volunteers changed children’s learning environment relative to what they would have ex- 
perienced otherwise (the service contrast created by the Jumpstart model). 


Although the study was able to examine whether certain features of the model were im- 
plemented as intended, there were three key data limitations to consider. First, while it was 


°The items in the survey were adapted from the Early Language and Literacy Classroom Observation 
(ELLCO) Pre-K Tool. 


possible to measure the amount of time that volunteers spent serving classrooms outside of 
Jumpstart session time, it was not possible to measure how much of the volunteers’ time was 
spent engaging with their partner children one-on-one during CCT (as opposed to providing sup- 
port to all children in the class). Second, volunteers’ use of teaching strategies focused on lan- 
guage and literacy development, as reported by teachers, may be biased. For example, given the 
nature of self-reporting, teachers may have overreported the extent to which they used these teach- 
ing strategies and underreported how often volunteers used them. Finally, the response rate for 
the Jumpstart post-service survey — which was used to measure volunteers’ impressions of the 
program — was only 45 percent, which is below the 70 to 80 percent range that is recommended 
for generalizing results to a broad population. 


Overall, the findings from this study indicate that the components of the Jumpstart model 
were generally implemented as intended, although limitations in the data make it challenging to 
assess whether that was accomplished with the expected level of quality: 


e Preparation and Training: The Foster Grandparents had consistently high 
attendance at both the pre-service and in-service trainings and reported feeling 
adequately prepared for providing services to children. 


e Center-Based Volunteer Teams: Teams of four to seven Foster Grandpar- 
ents were successfully assigned to each center as expected, and the volunteers 
demonstrated a strong commitment to the program. Foster Grandparents had 
consistently high attendance rates and most remained in the program for the 
entire year. Not all site managers offered regular feedback meetings as in- 
tended by the model; nonetheless, volunteers consistently said that their teams 
were effectively supported by the site managers and center staff. 


e Jumpstart Sessions: The Foster Grandparents implemented the Jumpstart 
sessions regularly and structured their delivery of the lesson plans as intended. 
The volunteers were also observed using various instructional practices aimed 
at improving children’s language and literacy development. The frequency 
with which these practices were used in the classroom varied across centers, 
with some practices used more consistently than others. 


e Child-Centered Time (CCT): The Foster Grandparents devoted a high num- 
ber of hours to CCT, more than the four to five hours a week expected in the 
Jumpstart model. However, it is not possible to determine how much of that 
time was spent one-on-one with partner children. More generally, the volun- 
teers were observed using recommended language and literacy strategies dur- 
ing one-on-one CCT, but strategies were not used consistently in all class- 
rooms. 


e Service Contrast: Because the Foster Grandparents dedicated a large amount 
of time to the Jumpstart program, children experienced an increased number 
of adults in these classrooms for a substantial portion of the week. During the 


sessions, teachers reported that the volunteers used language and literacy prac- 
tices almost as often as they did. This suggests that the teachers positively 
viewed the volunteers’ contributions, though it is not possible to determine 
whether this translated into more individualized adult-child interactions. Dur- 
ing CCT, while language and literacy instructional strategies were not used 
consistently across classrooms, teachers did report that volunteers engaged in 
a variety of one-on-one activities with children, providing individualized at- 
tention the children might not have received otherwise. 


Effects on Children’s Development 


Given the goals of the Jumpstart model, the impact study focused on two children’s outcomes: 
English Language and Literacy Development, and Social-Emotional Development. These out- 
comes were measured in the fall and in the spring, based on composite scores created from teach- 
ers’ ratings using the DRDP-PS. 


Even though a random assignment research design was used, the study has three im- 
portant limitations that make its findings challenging to interpret. First, teachers were not ran- 
domly assigned to classrooms, which means that the effect of Jumpstart could be confounded 
with differences in teacher characteristics or the instruction provided by teachers across class- 
rooms. Second, the number of children included in the analysis is small (105). Given this sample 
size, the estimated effect of the JFG model would have to be much larger than effects found in 
prior studies of the Jumpstart model — and larger than the effect of most educational interven- 
tions — to be statistically significant. Thus, the impact study is not powered to statistically detect 
effects of a reasonable magnitude. Third, because the DRDP-PS is used for purposes other than 
evaluation, child ratings from this tool may not accurately measure children’s true level of devel- 
opment. Given these limitations, the study’s findings should be considered exploratory and not 
used to make definitive conclusions about the Jumpstart model’s effect. 


Nonetheless, the potential for effects can still be explored by examining whether the find- 
ings are in the right direction and their magnitude. In this regard, the findings suggest that children 
in classrooms served by the Foster Grandparents did make gains during the year in their language 
and literacy skills and their social-emotional competencies. However, they did not make greater 
gains than children in the business as usual classrooms. 


Table ES.1 presents the average spring DRDP-PS scores of children in the Jumpstart 
classrooms and the business as usual classrooms. As shown in this table, children in the Jumpstart 
classroom had a spring DRDP score of about 6 points on their English language and literacy 
development, while children in the business as usual classroom had a score of 6.5 points. Thus, 
the estimated effect of the full Jumpstart model on children’s English language and literacy scores 
is numerically negative but small in magnitude and not statistically significant (effect size = -0.24, 
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p-value = 0.133). For social-emotional development, the average spring score of both groups of 
children was 7.7. Thus, the estimated effect on social-emotional scores is numerically positive 
but small in size and not statistically significant (effect size = 0.01, p-value of 0.951). 


Lessons About Implementing Volunteer-Based Intergenerational 
Programs 


The findings from this study of the Jumpstart Foster Grandparent model highlight several lessons 
about the implementation of volunteer-based educational interventions: 


e Itis possible to build a corps of senior volunteers who will commit signif- 
icant amounts of time and energy to providing services to children. Even 
though the volunteers were older adults — many of whom faced transporta- 
tion, health, and social barriers — the Foster Grandparents attended many 
hours of pre-service and in-service training and they also met, and sometimes 
exceeded, their service hour requirement of 15 hours a week. 


e Some instructional practices and strategies are more challenging for vol- 
unteers to implement than others. The findings from this study suggest that 
although the JFG volunteers implemented the language and literacy strategies 
that they learned in training, the frequency with which they were used varied 
across strategies and across centers. 


e High child mobility and sporadic attendance in early childhood education 
settings can make it challenging for staff to build relationships with chil- 
dren, which highlights the programmatic importance of child-centered 
strategies. In this study, many children left the centers part-way through the 
academic year and were not assessed in the spring. Children with inconsistent 
or unstable school attendance in preschool miss out on instruction and supports 
that will prepare them for kindergarten. Child mobility also makes it more 
challenging for adults in early childhood education centers to build relation- 
ships with children. Thus, organizations like Jumpstart that focus on relation- 
ships need to work especially hard to make sure that one-on-one time with 
children (like CCT) is maximized during the short time that children might be 
enrolled at the center. 


Looking Forward 


In summer 2019, Jumpstart decided to discontinue its Foster Grandparent program, which repre- 
sented about 4 percent of its total volunteer corps. This shift will allow Jumpstart to refocus its 
resources on the College Corps, a group that is better aligned with Jumpstart’s goal of training 
volunteers who will later become educators and teachers in the country’s educational workforce. 


Jumpstart has made several enhancements to CCT and to the Jumpstart sessions, which 
its College Corps implemented in fall 2019. The goal of these enhancements is to help the 


volunteers provide more intentional and focused language and literacy and social-emotional sup- 
ports to children. Volunteers are now trained on a modified version of CCT called Individual 
Classroom Service (ICS), to help them provide their partner children with higher-quality individ- 
ualized classroom time. ICS focuses on more intentional activities to promote language and liter- 
acy development. Volunteers receive training and support in the use of specific strategies for en- 
gaging children in conversations during one-on-one time, as well as activities for building 
vocabulary and comprehension skills. During ICS, volunteers continue supporting the general 
classroom by providing assistance to children during teacher-led activities, but there are guide- 
lines that emphasize how much time to dedicate to partner children. With respect to the Jumpstart 
sessions, the curriculum has been adapted to more strongly emphasize oral language and social- 
emotional skill-building. The sessions previously focused on phonological awareness, books and 
print knowledge, and oral language. Now the main focus is on oral language skills of vocabulary 
and comprehension, and the curriculum incorporates a new emphasis on supporting children’s 
ability to recognize, label, and understand emotions in themselves and others. These modifica- 
tions, which are intended to help the volunteers provide children with a substantially different 
instructional and supportive experience, have the potential to further strengthen the Jumpstart 
model and lead to better outcomes for children in the communities served by the program. 


Chapter 1 


Introduction 


Children’s school readiness as they enter kindergarten is an important predictor of their academic 
success and ultimately their long-term health and economic outcomes.! Research shows that par- 
ticipation in high-quality early childhood education helps preschoolers gain the foundational lan- 
guage and literacy skills as well as the social-emotional competencies they need to be ready for 
kindergarten.” In recent years, there have been efforts at the local, state, and federal levels to 
increase investment in publicly funded early education. Even so, free or affordable high-quality 
preschool options remain limited for families in many underserved communities, and children 
growing up in poverty often have less access to the kinds of early learning opportunities that 
contribute to school readiness.* Gaps in language and literacy skills begin early and widen over 
time.* Furthermore, a significant number of kindergarten teachers report that at least half of the 
children in their classes have problems with the social skills they need to succeed in school;> this 
is especially true in low-income communities. ° 


Jumpstart is a nonprofit organization that aims to address this readiness gap. It partners 
with early childhood education centers across the country to increase their capacity to provide 
high-quality language and literacy instruction to children in underserved communities.’ 
Jumpstart’s program model, delivered by volunteers, includes weekly curriculum-based sessions 
focused on children’s language and literacy development (“Jumpstart sessions”), as well as one- 
on-one time outside of those sessions (“child-centered time” or CCT). Most Jumpstart volunteers 
are college-age students (“College Corps”), but in some cities the volunteers have been older 
adults from the community (“Community Corps” or “Foster Grandparents,” depending on the 


city).® 


This report presents the findings from an evaluation of Jumpstart’s Foster Grandparent 
Program (JFG) in Los Angeles and Compton, California, which operated from fall 2011 to sum- 
mer 2019, where Jumpstart’s services were delivered by older adults. The evaluation, which fo- 
cuses on the 2017-2018 academic year, examines whether the JFG program was implemented as 
intended and to what extent the volunteers enhanced children’s educational and developmental 
experience in the classroom. The evaluation also explores whether the JFG program model shows 


‘Blodgett and Houghten (2018); Pace, Luo, Hirsh-Pasek, and Golinkoff (2017). 

Bettencourt, Gross, and Ho (2016); Brown (2014); Pre-Kindergarten Task Force (2017). 

3U.S. Department of Education (2015). 

4Fernald, Marchman, and Weisleder (2013). 

Raver (2002). 

6Manz and McWayne (2004). 

7Jumpstart partners with higher education institutions, community organizations, Head Start programs, com- 
munity-based preschools, and school districts. 

According to Jumpstart yearly reports, 4,054 Jumpstart volunteers served a total of 13,035 preschool chil- 
dren across the country in 2017-2018 (Jumpstart, 2019). 


promise for improving children’s language and literacy and social-emotional development, which 
includes children’s expression and management of emotions as well as the ability to establish 
positive and rewarding relationships with others (that is, interpersonal skills). The Corporation 
for National and Community Service (CNCS) funded the evaluation. 


This evaluation aims to build evidence about programs like Jumpstart that mobilize vol- 
unteers to provide services in their communities. Many programs across the country use volun- 
teers. For instance, through its Senior Corps program, CNCS supports organizations that connect 
seniors with opportunities to volunteer.’ The findings and lessons from this study may be helpful 
to policymakers and practitioners who work with volunteer-based programs, and to foundations 
and agencies like CNCS that provide such groups with financial support. This study also aims to 
inform policy and practice related to early childhood education. 


This chapter begins by providing an overview of the JFG program: its components, its 
delivery, and findings from prior studies about its effectiveness. The chapter then describes the 
study design and the data that were used in this evaluation to examine its implementation and its 
potential for improving children’s outcomes. Chapter 2 examines whether the Jumpstart model 
was implemented as intended, and to what extent the presence of volunteers enhanced the educa- 
tional experience of children in the centers. Chapter 3 explores whether the Jumpstart model as 
delivered by the Foster Grandparents shows promise for further improving children’s language 
and literacy and social-emotional development. Chapter 4 concludes by summarizing the main 
findings and discussing their implications for early childhood education programming and re- 
search. 


Jumpstart Foster Grandparent Program Overview 


Jumpstart is a national organization that works with early childhood education centers across the 
country. Jumpstart’s national office is in Boston and the organization’s national staff provides 
support and broad oversight related to curriculum design, volunteer recruitment, and evaluation 
and monitoring. Jumpstart also has offices across the country (including in Los Angeles) where 
staff members support the implementation of the programs. This includes site managers who are 
responsible for overseeing the implementation of the model in several preschool classrooms at 
one or more centers. In the Foster Grandparent program, three Jumpstart site managers oversaw 
volunteer placements and program delivery in 13 preschool classrooms in California. 


Jumpstart site managers recruited Foster Grandparents each summer by giving presenta- 
tions at senior centers, groups that work with seniors, and other sources such as faith-based and 
community organizations. During this recruitment period, Jumpstart staff also contacted seniors 


°The Senior Corps currently includes two programs: the RSVP program, where seniors can use their skills 
in a variety of settings, including teaching English to immigrants and helping communities rebuild after national 
disasters; and Senior Companions, where volunteers help adult clients live independently in their own homes. 
The Foster Grandparent program focused on recruiting senior volunteers to work with children with exceptional 
needs (Corporation for National and Community Service, n.d.). 


who had reached out to the organization during the year to inquire about volunteering. Jumpstart 
conducted an initial screening to determine eligibility for the JFG program. In order to become 
volunteers, interested seniors were asked to submit an application and an income eligibility 
form.!° Eligible seniors who met the income and commitment requirements (at least 15 hours per 
week) were invited to an interview with Jumpstart staff. '' The organization used the interview 
process to assess applicants’ ability to work in teams, their commitment to early childhood edu- 
cation, and their dedication to community service. Volunteers who had served in previous years 
were also invited to complete an abbreviated application and, if they met program expectations 
(for example, displayed professionalism and a commitment to early childhood education), were 
invited to volunteer for another year. 


Jumpstart offered Foster Grandparents several types of pre-service and in-service training 
to prepare and support them as they delivered the Jumpstart model. Volunteers were expected to 
participate in eight weeks of pre-service training (three days a week, six hours a day, totaling 
more than 138 hours). The pre-service training included an orientation about the Jumpstart cur- 
riculum, early childhood education, classroom management, and other relevant topics. The vol- 
unteers were also expected to attend monthly in-service trainings (four hours each), which the site 
managers facilitated. The managers covered areas that they had identified as needing additional 
focus based on their observations of the volunteers in the classroom, such as teamwork, mean- 
ingful conversations, and guided play with children. 


Each early childhood education center was assigned a team of four to seven Foster Grand- 
parents who supported it for the entire school year.'* Jumpstart strategically structured teams so 
they had a balance of new recruits and returning Foster Grandparents with varying levels of edu- 
cational attainment and experience in early childhood education. Each volunteer team was over- 
seen and supported by the Jumpstart site manager for that center, who observed and provided 
regular feedback to the volunteers. On days when volunteers were on-site, they were expected to 
meet after lunch for 1 to 1.5 hours to prepare and practice their delivery of the Jumpstart session 
plan for that week. On Fridays, the site managers also led a Team Leadership Meeting at a central 
location, where they reviewed and demonstrated the following week’s session plan to the volun- 
teer teams. At least two representatives from each team had to be present, but often the entire 
team attended the meeting. 


The overall goal of the JFG program was to place more trained adults in early childhood 
education classrooms so the children could receive more individualized attention, with a focus on 


‘0This application was several pages long and included questions about biographical information, availabil- 
ity and transportation needs, references, and placement requests. It also included open-ended questions about 
volunteers’ personal interests and goals, their experiences with and interest in children’s literacy, and their ability 
to work on diverse teams. The income eligibility form was shorter and included detailed questions about appli- 
cants’ recent and projected income. 

'lIncome could not exceed 200 percent of the poverty level to conform to the Edward M. Kennedy Serve 
America Act of 2009 (Jumpstart Senior Corps Income Guidelines, 2019). 

The number of volunteers in each center was based on the goal of having one volunteer for every three 
children during the Jumpstart session. 


their language, literacy, and social-emotional development. The Foster Grandparents provided 
this attention in two ways: Jumpstart sessions and CCT. 


Jumpstart sessions are based on a language and literacy curriculum designed to supple- 
ment existing preschool curricula.'* The sessions are intended to create opportunities for the vol- 
unteers to interact with children in a more structured format. At the time of the study, the 
Jumpstart sessions targeted the development of three areas of foundational language and literacy 
skills in children: oral language (vocabulary, comprehension); book and print awareness (alpha- 
bet knowledge and use of print); and phonological awareness (phonemic and rhyme). The content 
of the curriculum, as well as the supportive interactions between children and the Foster Grand- 
parents as they engaged in curriculum-based activities, were also intended to support children’s 
social-emotional development. 


For the JFG program, Jumpstart sessions were offered in a designated classroom in each 
early childhood education center.'* (This classroom will be referred to as the “Jumpstart class- 
room” throughout this report). Two mornings a week, the volunteers led a structured, two-hour 
session with the children in that classroom. The Jumpstart sessions took place during regularly 
scheduled classroom time and were led by the volunteers. The lead preschool classroom teacher 
and the teaching assistants were also present; their role was to monitor behavior, encourage chil- 
dren to take part, and participate along with the children and volunteers. As a result, the number 
of adults in the Jumpstart classroom was significantly increased during session time. 


The Jumpstart curriculum was delivered in 40 sessions designed to provide children with 
a balance of individual and group learning activities that facilitated interactions with the volun- 
teers and enhanced the individualized attention the children recetved. Each session followed the 
same routine to support the participation of children and adults during Jumpstart time. Those 
session elements were: Welcome, Reading, Circle Time, Center Time, Let’s Find Out About It, 
and Sharing and Goodbye (see Table 1.1 for a detailed description of session segments). Each 
session revolved around a core storybook that connected to an overall unit theme and served as 
the focus for many of the learning activities included in the session.'> Each storybook also in- 
cluded social-emotional content that supported children’s development of interpersonal skills. 
During Jumpstart session time, volunteer interactions were individualized and further promoted 
by pairing each Foster Grandparent with three to four partner children whom they supported dur- 
ing small group activities throughout the entire year (during the Welcome and Reading segments 


'3Jumpstart’s curriculum was adapted from the Opening the World of Learning (OWL, v. 2005) curriculum. 

'4Tn a center, sessions were scheduled for the same two days every week, either Monday and Wednesday or 
Tuesday and Thursday. Although most centers have more than one classroom, JFG sessions were delivered in 
only one classroom. This classroom did not change throughout the year. 

'SThere were six units: The Family; Friends; Wind and Water; The World of Color; Shadows and Reflec- 
tions; and Things that Grow. 


Segment 


Format 


Table 1.1 


Jumpstart Session Segments 


Overview 


Duration 
(minutes) 


Welcome 


Small group 


The volunteers transition children to the 
Jumpstart session by involving them in alphabet 
knowledge and phonemic awareness activities 
with their names. 


2-5 


Reading 


Circle Time 


Small group 


Full group 


Volunteers present and read the book of the day 
to introduce vocabulary and engage children in 
discussions that support comprehension. There 
are two parts to the reading: (1) reading for en- 
joyment, and (2) reading to reconstruct (inter- 
spersing the reading with questions about the 
text). 


The children and volunteers join 

together in four group activities such as singing 
songs or playing games that support skill devel- 
opment. 


15 


15 


Center Time 


Let’s Find Out 
About It 


Sharing and 
Goodbye 


Small group 


Small group 


Full group 


The team leader introduces the activities at five 
activity centers. Volunteers disperse to each cen- 
ter and interact with children in a variety of activi- 
ties. The centers are: 

e §=©Writing 

e Books 

e Puzzles and manipulatives 

e Dramatic play 

e Art or science 
Children choose which center they want to attend 
first, then circulate through the others as much 
as they want. They don’t have to participate in all 
centers. 


There is also a “Let’s Find Out About It” activity 
facilitated at the beginning of Center Time. This 
activity allows children to dive deeper into a con- 
cept from the book of the day or from the unit 
theme. Only half of the children participate in this 
activity; the other half gets to do it in the second 
session implementation. After the small group 
finishes this activity, they can join and circulate 
through the centers of their choice. 


Volunteers ask four children to share what they 
learned during the day and reflect on the lesson. 
Volunteers then share a summary of the next 
session and say goodbye. 


50-55 (includ- 
ing 5-minute 
introduction) 


15 


of the sessions).'° The assignment of children to volunteers for Jumpstart sessions was intended 
to remain consistent throughout the year. 


The volunteers were provided with a series of 20 session plans, along with the materials 
needed for the activities. Each session plan provided detailed information about activities to en- 
gage the children, how to implement those activities, and why they were important. Foster Grand- 
parents were expected to deliver Jumpstart sessions twice a week for 20 weeks (each storybook 
was covered twice), for a total of 40 sessions during the school year. During the second imple- 
mentation of each session plan, children continued to develop an interest in and enjoyment of the 
stories, while also deepening their understanding of target vocabulary, story events and their 
causes (comprehension), and related content knowledge. During each session, one volunteer 
served as the activity leader, guiding the classroom through the plan and facilitating transitions 
between activities. Some teams rotated the role of activity leader, while in other teams the role 
was assumed by the Foster Grandparent who felt most comfortable taking on that responsibility. 


While the Jumpstart sessions were intended to provide a structured format for volunteer- 
child interactions, the second component of the JFG model — child-centered time — consisted 
of time outside of the formal sessions, where volunteers interacted with children one-on-one in a 
less-structured format. As an integral part of service, CCT gave volunteers opportunities to inter- 
act with their partner children and contribute to their language and social-emotional development 
within the setting of teacher-led instructional time. A Foster Grandparent might support partner 
children during class circle time, for example, or they might help facilitate learning during child- 
initiated activities.'’? The nature of these interactions between Foster Grandparents and partner 
children varied from classroom to classroom and from volunteer to volunteer. Some teachers 
provided suggestions and directions to volunteers on how to support partner children, while other 
teachers allowed Foster Grandparents more flexibility and encouraged them to introduce new 
concepts and activities that were related to Jumpstart’s language and literacy domains. '* None- 
theless, an integral part of the CCT model was that it be completely adaptable to the classroom’s 
usual routine. Each volunteer was expected to provide four to five hours of CCT throughout the 
week, according to their availability and classroom teachers’ preferences. Foster Grandparents 
played important roles in the classroom: by building strong relationships with children; increasing 


‘©The strategy for grouping and assigning children to volunteers was often left to the discretion of classroom 
teachers and center staff and hence varied across centers. For example, some centers allowed volunteers to work 
with all the children in the classroom where Foster Grandparents delivered services and the teacher later assigned 
children to volunteers based on bonds that formed organically. In other centers the teachers made assignments 
based on the needs of the children. 

‘Tn child-initiated activities, children were free to move from one activity to another as they felt motivated. 
They were encouraged to make their own activity choices, take out the appropriate materials, and put them away 
when they were done. In a Jumpstart classroom, the teacher’s role was to arrange the environment and provide 
ideas and materials for children to choose their own activities. 

'8Prior to the start of the program year, Jumpstart staff and teachers participated in an orientation to build a 
shared understanding about each other’s roles and establish systems and processes for a successful school year, 
including CCT schedules, attendance policies, and requirements and expectations for implementing Jumpstart. 
Throughout the year, teachers received week-by-week curriculum highlights. 


the number of adults and thereby increasing opportunities for children to receive individualized 
attention (from both volunteers and teaching staff); and by supporting children’s participation and 
learning during activities planned by classroom teachers. Foster Grandparents were also expected 
to build collaborative relationships with teachers and support teachers in the classroom. 


The Foster Grandparents provided CCT in the Jumpstart classroom and other classrooms 
at the early childhood education centers. In the Jumpstart classroom, the volunteers provided CCT 
to their partner children outside of session time; in the other classrooms at the center (CCT-only 
classrooms), a subset of Foster Grandparents also provided CCT to an assigned group of one to 
three children.'? The criteria used by teachers to select which children got CCT in these class- 
rooms differed across centers but could include: a child’s developmental needs (children who 
required extra help); a child’s age (younger children, because they needed additional support tran- 
sitioning from home to preschool, or older children, because they could benefit from more support 
as they prepared to transition to kindergarten); or a child’s attachment (volunteers were paired 
with the children who formed a bond with them). The assignment of partner children to volunteers 
was intended to stay relatively stable throughout the year. 


The volunteer teams typically provided CCT from early October to the end of May or 
mid-June, whereas the Jumpstart sessions began a bit later, in mid- to late October, and ended in 
mid-May. As already noted, Jumpstart sessions were offered two mornings a week for two hours 
and CCT took place outside of session time. CCT occurred throughout the week, sometimes di- 
rectly before the Jumpstart sessions, but also on days when Jumpstart sessions were not con- 
ducted. Children in Jumpstart classrooms were expected to receive CCT for at least three hours a 
week outside of session time. In the CCT-only classrooms at centers, Foster Grandparents were 
also assigned to a group of one to three partner children who were expected to receive three hours 
of CCT a week. 


Figure 1.1 summarizes the key components of the Jumpstart model and its theory of 
change. As shown in this figure, the implementation of the JFG program components (and the 
resulting addition of trained adults in the classroom) was expected to set in motion several pro- 
cesses: (1) Volunteers enhanced classroom instruction by targeting children’s key foundational 
language and literacy skills (oral language, phonological awareness, and book and _ print 
knowledge); (2) volunteers provided one-on-one attention and support to children so that mean- 
ingful relationships between volunteers and children could develop; (3) there would be more op- 
portunities for children to be exposed to language and more opportunities for conversational ex- 
changes between children and volunteers; and (4) the volunteers would get to learn more about 
early childhood education and develop and practice new skills. This, in turn, would lead to im- 
provements in children’s language and literacy and social-emotional development. The specific 


‘The volunteers were required to provide CCT in other classrooms (and not just the Jumpstart classroom) 
for two logistical reasons: (1) During “regular” classroom time, if all volunteers provided CCT in the same class- 
room at the same time, there would be too many adults in the classroom; and by extension (2) Foster Grandpar- 
ents had to provide CCT in other classrooms to meet their time commitment. 
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program components (Jumpstart sessions and CCT) were theorized to have an impact on children 
in the following ways: 


e Effect of the Jumpstart sessions: The Jumpstart sessions were expected to 
have a positive effect on the language and literacy development of children in 
the classroom — more specifically, their oral language, phonological aware- 
ness, and book and print knowledge. Because the Jumpstart curriculum is in 
English, program effects were mainly expected on English-language develop- 
ment. In addition, the Jumpstart sessions might also improve children’s social- 
emotional development through the content of stories and activities and the 
increased opportunity to participate in structured group activities. Through on- 
going support and more frequent, high-quality interactions with the volunteers, 
children were also developing secure relationships with adults that could en- 
hance their emotional development. 


e Effect of CCT: Providing CCT in a classroom during nonsession time was 
expected to have a positive effect on the language and literacy and social-emo- 
tional development of children through two pathways. First, children’s lan- 
guage and literacy development and their social-emotional development 
would be enhanced through the one-on-one attention that they received from 
the volunteers. (In non-Jumpstart classrooms, this direct effect of CCT applied 
to the subset of children who were partnered with a volunteer.) Second, chil- 
dren’s development might also be improved through the benefits of having 
more adults in the classroom during nonsession time. More adults in the class- 
room could create more opportunities for teachers and teaching assistants to 
provide one-on-one attention to all children in the classroom. This indirect or 
secondary effect of CCT could benefit all children in the classroom, including 
those who were not directly paired with a volunteer. Because it could affect all 
children in the classroom, CCT could therefore be considered a classroom- 
level intervention component. 


e Effect of the full Jumpstart model: Finally, it was expected that the combi- 
nation of Jumpstart sessions with CCT (the full Jumpstart model) would pro- 
duce even greater benefits than offering these services in isolation. In the full 
Jumpstart model (which was provided in the Jumpstart classroom), there were 
multiple opportunities for volunteers to build a strong relationship with their 
partner children: during the small group activities that took place during the 
session, and again though one-on-one CCT. The stronger relationships that 
were created through these two types of activities might increase the effect of 
CCT as well as the effect of the sessions, beyond what would be if they were 
offered in isolation. 


Although the effect of the Jumpstart model as implemented by senior volunteers has not 
previously been externally evaluated, prior studies of the model as implemented by the College 


Corps found promising effects on children’s outcomes. These studies found that children who 
received services from Jumpstart volunteers outperformed comparison children on measures of 
early literacy (expressive vocabulary, target vocabulary, and oral literacy), school readiness, and 
social-emotional skills.*° Moreover, a substantial body of research suggests that intergenerational 
models have the potential to positively affect the outcomes of the children involved, including 
their mental health (reduced anxiety, improved self-worth) and their attitudes toward older 
adults.*! Recent studies have also shown that educational interventions delivered by volunteers of 
varying ages can improve the educational outcomes of children from low-income families. For 
example, a recent study of Reading Partners — a tutoring program that mobilizes volunteers who 
are not certified teachers — found that it had a positive and statistically significant impact on the 
reading achievement of elementary school students.” 


It was further expected that the volunteers themselves would benefit from participating 
in Jumpstart. Volunteers would learn more about early childhood education and develop new 
skills in a wide range of areas, such as working in teams, communication, and problem-solving. 
They would also create meaningful connections with children and watch them grow. This could 
help the volunteers to be more effective in their role of improving children’s development, while 
at the same time, improving their own well-being, including better overall physical health and 
lower levels of depression and isolation. Findings from a recent independent study showed that 
first-time volunteers in the Foster Grandparent and Senior Companion programs (both funded by 
CNCS) reported improvements in physical and mental health after two years of service.” Ulti- 
mately, intergenerational programs appear promising for both children and older adults, with 
wide-ranging effects. 


Overview of the Evaluation 


The purpose of this evaluation is to expand the body of research on the Jumpstart model, focusing 
on how the model was delivered by Foster Grandparents. Many interventions rely on volunteers 
to provide services to children and youth, so this study also aims to build broader knowledge 
about the implementation and potential effects of educational interventions that mobilize volun- 
teers to provide services to children, especially ones like Jumpstart that require a relatively large 
time commitment. The findings from this study can also inform policy and practice about the 
potential benefits of intergenerational programs. 


This evaluation of Jumpstart includes an implementation study and an impact study. The 
goal of the implementation study is to describe the content, quantity, quality, and structure of 
services that the Foster Grandparents provided to children in the centers in this study. 


Harris and Berk (2011); Dwyer (2012); Grant and Shannon (2015). Two of these studies used a quasi- 
experimental research design and one used a child-level random assignment research design. 


21Park (2015). 
22 Jacob, Armstrong, and Willard (2015). 
3Georges et al. (2018). 
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Accordingly, the study examines the following questions related to the implementation of the 
JFG program: 


e To what extent were the key Jumpstart program components (volunteer train- 
ings, center-based volunteer teams, Jumpstart sessions, and CCT) put in place 
and implemented with the intended frequency, intensity, and duration? 


e How many hours of Jumpstart sessions and CCT did children receive on aver- 
age during the academic year? 


e To what extent did the training and the program materials prepare the volun- 
teers to implement the program? 


In addition to how well the program was implemented, another important driver of 
Jumpstart’s effects on children is the extent to which the presence of the volunteers affected the 
classrooms. The implementation study examines questions related to the service contrast created 
by the Jumpstart model: 


e Towhat extent were there more adults in classrooms served by the volunteers 
than in classrooms not served by the volunteers? 


e Did children in classrooms served by the volunteers experience more individ- 
ualized interactions promoting their language and literacy development than 
children in classrooms not served by the volunteers? 


This evaluation of Jumpstart also includes an impact study. The goal of the impact study 
is to explore whether the full Jumpstart model and its components, as delivered by the Foster 
Grandparents, show promise for improving children’s outcomes: 


e What was the effect of providing the full Jumpstart model (sessions and CCT) 
on children’s language and literacy development and their social-emotional 
development? 


e What was the classroom-level effect of providing CCT on children’s language 
and literacy development and their social-emotional development? 


As will be described in greater detail in this section, the effect of the Jumpstart model was 
examined by randomly assigning children to classrooms that received different levels of services 
from the volunteers (full Jumpstart model, CCT-only, or no services). The remainder of this chap- 
ter describes the implementation and impact studies in greater detail, including the centers that 
participated in the evaluation, the random assignment of children to classrooms, and the data 
sources and measures that were used to measure implementation and child outcomes. The limi- 
tations of the evaluation are also discussed. 
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Recruitment of Study Centers 


Thirteen early childhood education centers in Los Angeles and Compton, California, im- 
plemented the JFG model in 2017-2018, and all of them were approached about participating in 
the study. To do so, they had to be willing to: (1) share data that they already collect about enrolled 
children (for example, teacher reports, daily attendance, etc.); (2) help with obtaining parental 
consent for children to take part in evaluation activities and for the study team to view the chil- 
dren’s records; and (3) allow the study team to randomly assign all or a subset of children to 
classrooms in the 2017-2018 academic year. Eleven centers agreed to the first two criteria and 
are included in the implementation study (these centers will be referred to as the “study centers” 
in this report). Of these eleven, five centers agreed to the random assignment of children to class- 
rooms and are included in the impact study (these centers will be referred to as the “impact study 
centers” in this report). 


The centers in the study are operated by a total of six umbrella organizations or agencies 
that offer a variety of social services in their local communities. Most of them are Head Start 
centers where children can attend all day. One center offers half-day classes, which means it 
serves different groups of children in the morning and afternoon. All serve children from low- 
income families, which reflects the population served by Jumpstart in the Los Angeles area.” 
Four centers have only one classroom of preschool-age children, while the others have more than 
one classroom (see Table 1.2). All classrooms included in the study were mixed-age classrooms 
(3- and 4-year-olds). 


Random Assignment and Recruitment of Children 


In this study, the impact of the JFG model on children’s development is explored using a 
three-group random assignment research design. In each of the five centers in the impact study, 
children were randomly assigned to one of three types of classrooms: 


e Jumpstart classroom: A classroom that received the full JFG model, where 
all children received the Jumpstart sessions and child-centered time outside of 
the Jumpstart sessions; 


e “CCT-only” classroom: A classroom where children did not receive the 
Jumpstart sessions but where some children received child-centered time; and 


e “Business as usual” classroom: A classroom where children did not receive 
any Jumpstart services from the volunteers. 


4Tn 2017-2018, children served by the Jumpstart Foster Grandparent program in Los Angeles had the fol- 
lowing characteristics: About 96 percent were 3 or 4 years old (the remainder were 2 or 5 years old or older); 75 
percent were Latino and 15 percent were black; 52 percent were boys (48 percent girls); 82 percent were dual 
language learners; 59 percent were most comfortable speaking English; and 38 percent were most comfortable 
with Spanish. 
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Table 1.2 
Jumpstart Foster Grandparent Study Agencies 


Agency # of centers # of classrooms 
Center 1 3 
Agency 1 Center 2 3 
Center 3 3 
Center 1 3 
Agency 2 Center 2 4 
Center 3 1 
Agency 3 Center 1 8 
Agency 4 Center 1 1 
Agency 5 Center 1 1 
Center 2 1 
Agency 6 Center 1 3 


Only children who were new to the centers were randomly assigned to classrooms, be- 
cause centers preferred that returning children stay with their teachers from the previous year. 
Random assignment was conducted separately for each center, and in some places also by special 
needs, by age, and/or by gender. This “blocking” of random assignment helped ensure that class- 
rooms in each center were similar in terms of age distribution, gender, and needs.”> Random as- 
signment was conducted in summer 2017. 


Using this study design, the impact of the full JFG model could be estimated by compar- 
ing the spring outcomes of children in the Jumpstart and business as usual classrooms. Similarly, 
the classroom-level impact of CCT could be estimated by comparing the spring outcomes of chil- 
dren in the CCT-only and the business as usual classrooms. 


An important advantage of this study design is that the random assignment of children to 
classrooms ensured that children in the three types of classrooms were at similar development 
levels in the fall before the Jumpstart program was delivered. This means that any observed dif- 
ferences in children’s outcomes in the spring were not confounded with preexisting differences 
between the three groups. However, as will be discussed later in this chapter, teachers were not 
randomly assigned to classrooms. 


Data Sources 


Several types of data were used in this evaluation (see Table 1.3). For the implementation 
study, various sources of information were used or collected to assess the delivery of the Jumpstart 


5Centers expressed that this balance is important to them. In addition, blocking can also improve the preci- 
sion of impact estimates and statistical power. 
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Table 1.3 


Overview of Data Sources 


Data Source 


Data Collection Timing 


Domains 


Implementation Study 


Session attendance logs 


Throughout program year 


Consistency of Jumpstart sessions 
offered 


Attendance during Jumpstart sessions 


Jumpstart session and 
team planning checklists 


A couple of times per 
program year 


Implementation of structured session 
components 


Training attendance 
records 


Pre-service and monthly in- 
service during program year 


Participation in trainings 


Volunteer timesheets 


Volunteer pre-service 
survey 


Throughout program year 


Fall 


Volunteer team participation in program 
and overall retention rate 


Volunteer presence during Jumpstart 
sessions and CCT 


Hours of Jumpstart sessions and CCT 
offered 


Volunteer characteristics 


Volunteer post-service 
survey 


Spring 


Volunteer preparation for service after 
training 


Usefulness of supervision, feedback, and 
meetings 


Relationship between volunteer and pre- 
school staff 


Volunteer interviews 


Spring 


Training quality 


Volunteer team quality 


CCT quality 


Volunteer knowledge and skill gains 


Teacher survey 


Spring 


Language and literacy instruction 
observed during Jumpstart sessions 


Language and literacy instruction 
observed during CCT 


Classroom management 


Teacher characteristics 


Impact Study 


Parental consent form 


Fall 


Child and family characteristics 


DRDP-PS ratings 


Fall and spring 


English Language and Literacy 
Development 


Interpersonal Skills 


Self-Regulation 
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model in the study centers. To begin, the study team relied on data sources collected by Jumpstart 
to internally monitor program implementation during the 2017-2018 academic year: 


e Session attendance logs: As part of Jumpstart’s internal monitoring system, 
the volunteers collect data on the attendance of children during the Jumpstart 
session. The study team used these data to measure the amount of services 
(session dosage) the children received and to assess the amount of time volun- 
teers committed to the program and the children. 


e Session and Team Planning Checklist: Jumpstart site managers regularly 
use an observational tool that helps them keep track of whether the volunteers 
are using the specified plan activities. The study team used data from the 
checklists to examine whether the volunteers implemented the Jumpstart ses- 
sions as intended. 


e Training attendance records and volunteer timesheets: Jumpstart records 
the attendance of volunteers at the pre-service trainings and tracks the volun- 
teers’ hours served using monthly timesheets. The timesheets capture training 
time, the number of hours the volunteers spend at the centers each day across 
the entire year, and whether they are providing sessions, participating in team 
planning meetings, or providing nonsession support. Given the volume and 
complexity of the timesheet data, the study team randomly sampled 16 volun- 
teers and analyzed their monthly timesheets for the 20 weeks in 2017-2018 
that Jumpstart sessions were implemented in the study centers.”° These data 
were used to measure the amount of time volunteers spent on-site and what 
they did during that time. 


e Volunteer survey: Jumpstart administers a pre-service and a post-service sur- 
vey to its volunteers. Data from Jumpstart’s fall 2017 volunteer survey were 
used to examine the characteristics of the volunteers in the study centers, and 
data from the spring 2018 survey were used to examine volunteers’ percep- 
tions of the training and the program. 


To supplement these existing data sources, the study team also collected additional data 
on implementation specifically for the purposes of the study: 


e Volunteer interviews: During the JFG program’s final team leadership meet- 
ing (May 2018), the study team conducted one-on-one interviews with 12 vol- 
unteers. (One participant was randomly selected from each volunteer team.?’) 
Volunteers were asked about the training provided by Jumpstart, the 


6The timesheets contained extensive amounts of data that were heavily formatted, which made extracting 
information from the timesheets challenging. 


?7 All volunteers in all 11 study centers were present for this training and everyone had an equal opportunity 
to be chosen for an interview. The duration of these interviews was 15-20 minutes. 
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functioning of their center-based team, their day-to-day practices, and how 
they benefitted from being a volunteer. 


e Teacher survey: Lead teachers were given a survey to assess the level at 
which high-quality language and literacy practices were used in their class- 
rooms — by themselves or the volunteers.*® Some questions in the survey were 
adapted from the Early Language and Literacy Classroom Observation 
(ELLCO) Pre-K Tool, a well-established observational tool designed to cap- 
ture the quality of language and literacy instruction in preschool classrooms.” 
All teachers*° responded to the survey. 


Two data sources were used for the impact study. The first was a short intake survey that 
parents and guardians were asked to complete during the parental consent process.*! The survey 
included several items about the characteristics of their children and their families, and the infor- 
mation was used as covariates in the impact analysis. 


Also for the impact study, the study team measured children’s outcomes with a teacher- 
reported assessment tool that is used by all the study centers called the Desired Results Develop- 
mental Profile — Preschool. The DRDP-PS is an observational tool developed by the California 
Department of Education with support from researchers at University of California, Berkeley, 
that can be used by teachers to rate children’s development in several areas. Early childhood ed- 
ucation centers in California that receive state funding (including all 11 study centers) must use 
the DRDP-PS to assess each child’s development in the first 60 days of enrollment and then again 
after six months. Children’s skills are rated in eight domains and each domain includes several 
items.** For each item, there is a well-defined, nine-point scale representing different levels of 
development.** To improve rating consistency across teachers, each item is accompanied by a 
description of the behaviors and skills that children at each level exhibit on that item, and specific 
illustrative examples. When used correctly, domain-specific scales from the DRDP-PS are highly 
correlated with other established developmental measures, and its scales have been shown to have 
high internal consistency and inter-rater reliability. *4 


>8The teacher survey instrument can be found in Appendix A. 

>°Polk (2013). 

3°This number includes 27 lead teachers and 8 assistant teachers. Only lead teacher survey responses were 
analyzed. 

31Tn all study centers, children’s parents were asked for consent for the center to share information with the 
study team about their children. The consent process took place between August 2017 and mid-October 2017; 
73 percent of parents agreed to their child’s participation in the study (see Appendix B, Figure B.1). 

3?These domains are: Approaches to Learning--Self-Regulation; Social and Emotional Development (which 
measures children’s interpersonal skills); Language and Literacy Development (which rates children’s develop- 
ment in their main language); English-Language Development (for Dual Language Learners only); Cognition; 
Physical Development-Health; History-Social Science; Visual and Performing Arts. 

33The nine levels are: Responding (earlier and later), Exploring (earlier, middle, later), Building (earlier, 
middle, later), and Integrating (earlier). 

34DRDP Collaborative Research Group (2018). 
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For the purposes of the evaluation, the study centers provided DRDP scores for fall 2017 
and spring 2018 for all children whose parents agreed to the release of these data. The impact 
study focused on children’s scores in four domains measured by the DRDP that are highly aligned 
with Jumpstart’s theory of change: Language and Literacy, English-Language Development, In- 
terpersonal Skills,*> and Self-Regulation. The two former domains were used to create an English 
Language and Literacy Development measure.*° The two latter scores were combined into an 
overall measure of social-emotional development for the purpose of this study. (See Appendix C 
for more information on the items in each domain).*” 


Limitations 


Both the implementation study and the impact study have limitations that make it chal- 
lenging to interpret their results and to draw definitive conclusions about the potential effects and 
implementation of the Jumpstart model in the study centers. These limitations are discussed be- 
low. 


Implementation Study 


The main limitations of the implementation study are related to its data sources. The first 
limitation is that the study was not able to measure the amount of time the volunteers devoted to 
specific children during CCT. The volunteer timesheets tracked the number of hours that volun- 
teers spent serving classrooms, but do not drill down to reflect which children the members were 
supporting throughout different aspects of their service. The study team worked with Jumpstart 
staff to identify a set of assumptions to best estimate how many hours volunteers spent in sessions 
and in CCT across classrooms. However, because they rely on assumptions, any findings based 
on the timesheets should be interpreted with caution. Furthermore, as mentioned earlier, the re- 
search team did not analyze the timesheets of all volunteers (16 volunteers’ timesheets were ran- 
domly sampled for the analysis).** 


A second limitation of the implementation study relates to the difficulty of determining 
whether the presence of the volunteers in the classrooms increased the individualized attention 
the children received and improved the quality of adult-child interactions. Because lead teachers 


35In the DRDP, the Interpersonal Skills domain is referred to as the Social and Emotional Development 
domain. The research team renamed it for the purpose of this report. Social-Emotional Development is used in 
the report to refer to a composite measure of DRDP domains. 


3°The English Language and Literacy Development outcome that is used in this evaluation is based on chil- 
dren’s scores on two DRDP domains: For children who were not dual language learners, their English Language 
and Literacy Development score was based on their ratings on items in the DRDP’s Language and Literacy 
Development (LLD) domain. For children who were dual language learners, their score for the DRDP’s Eng- 
lish-Language and Literacy Development domain was used instead, because this assesses their progress to- 
ward learning to communicate in English (whereas the LLD score may reflect language and literacy development 
in achild’s home language). 

37Three of the seven measures in the Self-Regulation domain are conditional measures that were only scored 
when a child had an individualized education plan or was a dual language learner. These conditional measures 
were not included in the composite social-emotional learning measure. 


38The study team received timesheets for all 71 Foster Grandparents who volunteered at the study centers. 
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were present in the classroom at most times, a survey was used to ask them to report on the extent 
to which they and the volunteers used teaching strategies aimed at promoting children’s language 
and literacy development, as well as the types of activities that volunteers engaged in with chil- 
dren during one-on-one time. However, because this was based on self-reporting — as opposed 
to a report from an independent observer — it is possible that teachers overreported the extent to 
which they used these strategies or that they underreported how often volunteers were using these 
strategies. Thus, the teacher survey provides an imperfect picture of the instructional experience 
of children in the study classrooms. 


Finally, the response rate for the Jumpstart post-service survey — used to measure vol- 
unteers’ impressions of the program — was only 45 percent, below the 70 to 80 percent range 
that is recommended for generalizing results to a broad population. This means that findings re- 
lated to volunteers’ perceptions of Jumpstart trainings and the program, which will be discussed 
in Chapter 2, should be interpreted with some caution. 


Impact Study 


A key limitation of the impact study is its design. As mentioned earlier, teachers were not 
randomly assigned to classrooms (or, viewed otherwise, Jumpstart services were not randomly 
assigned to classrooms). Instead, since centers affiliated with the JFG program could choose 
which classroom would implement the Jumpstart sessions, that means there could be systemic 
differences in teacher characteristics or quality across the three types of classrooms, and these 
differences could be confounded with the effect of Jumpstart on children.*’ In the lead teacher 
survey mentioned earlier, teachers were asked about their characteristics, experience, and educa- 
tion. On average across classrooms, the lead teachers in this evaluation were highly experienced 
educators, with 14 years or more of early education teaching experience, and they all had at least 
some college education. There were also notable differences across the three classroom types 
(Jumpstart, CCT-only, and business as usual) with respect to teachers’ ages and their likelihood 
of having a bachelor’s degree.*° These differences, and other unmeasured differences, may have 
led to variation in the instruction received by children across classroom types (or, alternatively, 
variation in how these teachers rated children’s development on the DRDP). 


A second limitation of the impact study is its small sample size. The analysis sample for 
the impact study included children enrolled in the five impact study centers whose parents con- 
sented to their participation, and who had both fall and spring data on the DRDP-PS. The resulting 
analysis sample includes 105 children. Given this sample size, the estimated effect of the JFG 
model would have to be much larger than effects found in prior studies of the model — and larger 
than the effect of most educational interventions — to be able to conclude that Jumpstart’s effects 


°Centers make decisions about the Jumpstart classroom based on a range of factors. In some centers, the 
director asks for a volunteer; in others, the director chooses the classroom based on which teacher would most 
benefit from the help or, conversely, is skilled enough to integrate the sessions in his or her classroom. In another 
center, the Jumpstart session rotates across teachers each year. 


4°See Appendix F, Table F.1 and F.2. 
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were statistically significant.*' Thus, the impact study is not powered to statistically detect effects 
of a reasonable magnitude.” 


A third limitation of the impact study is that children’s outcomes were measured using 
an existing source of data (the DRDP-PS) instead of collecting child data specifically for the 
purposes of the study. This made it possible to measure children’s outcomes at a lower cost and 
lower burden for the study centers. However, the pattern of DRDP-PS scores for children in the 
study centers indicates that teachers may not have been using the tool as intended. For instance, 
as will be discussed in Chapter 3 of this report, the pattern of differences in DRDP-PS scores 
across classrooms suggests that teachers might have been interpreting the scales and ratings in 
different ways. This indicates that inter-rater reliability may be low for teachers in the study, 
which in turn could make it more difficult to produce effects on this measure in the study. 


For these reasons, the findings from the impact study should be considered exploratory 
and should not be used to make definitive conclusions about the impact of the Jumpstart Foster 
Grandparent model. 


Despite these limitations, the results from this evaluation can still provide a general pic- 
ture of how the Jumpstart model was implemented by Foster Grandparents in the study centers. 
The results can also be used to generate a set of lessons that may be useful to volunteer-based 
intergenerational programs that, like Jumpstart, aim to improve the outcomes of children from 
traditionally underserved communities. 


Report Roadmap 


The remainder of this report is structured as follows: Chapter 2 examines whether the 
Jumpstart model was implemented as intended and to what extent the presence of volunteers 
enhanced the educational experience of children in the centers. Chapter 3 explores whether the 
Jumpstart model, as delivered by the Foster Grandparents, shows promise for further improving 
children’s language and literacy and social-emotional development. Chapter 4 concludes by sum- 
marizing the main findings and discussing their implications for early childhood education pro- 
gramming and research. 


41The minimum detectable effect size (MDES) for this study is between 0.39 and 0.54. As a reference point, 
the effect of the Reading Partners intervention (which uses volunteers to tutor children) mentioned earlier in this 
chapter was 0.11 (Jacob, Armstrong, and Willard, 2015). 

“Because of the small sample size in the impact study, the research team conducted an additional explora- 
tory analysis to bolster the analytic sample. In this analysis, children in Jumpstart and CCT-only classrooms in 
study centers where random assignment did not happen were matched to similar children in business as usual 
classrooms. This quasi-experimental sample of children was added to the sample of randomized children. The 
pattern of findings based on this larger sample is similar to those from the randomized experiment and also 
inconclusive. (See Appendix E for details). 
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Chapter 2 


Implementation and Service Contrast 


This chapter examines the implementation of the Jumpstart Foster Grandparent program in the 
study centers during the 2017-2018 academic year. Four components of the program will be dis- 
cussed: volunteer preparation and training; the structure and deployment of center-based volun- 
teer teams; the Jumpstart sessions; and child-centered time (CCT). As explained in the previous 
chapter, several types of data were collected to explore whether these components were imple- 
mented as intended and to what extent the presence of the volunteers enhanced the experiences 
of children compared to what the children would have experienced otherwise. The key findings 
from the implementation study are: 


e Preparation and Training: The Foster Grandparents had consistently high 
attendance at both the pre-service and in-service training sessions and reported 
feeling adequately prepared for providing services to children. 


e Center-Based Volunteer Teams: Teams of four to seven Foster Grandpar- 
ents were successfully assigned to each center as expected, and the volunteers 
demonstrated a strong commitment to the program. Foster Grandparents were 
consistently present in classrooms and most of them remained in the program 
for the entire year. 


e Jumpstart Sessions: The Foster Grandparents implemented the Jumpstart 
sessions regularly and structured their delivery of the session plans as intended. 
The volunteers were also observed using various instructional practices aimed 
at improving children’s language and literacy development. The frequency 
with which these practices were used in the classroom varied across centers, 
with some practices used more consistently than others. 


e Child-Centered Time (CCT): The Foster Grandparents devoted a high num- 
ber of hours to CCT, more than the four to five hours a week expected in the 
Jumpstart model. However, it is not possible to determine how much of that 
time was spent one-on-one with partner children. More generally, the volun- 
teers were observed using recommended language and literacy strategies dur- 
ing one-on-one CCT, but strategies were not used consistently in all class- 
rooms. 


e Service Contrast: Because the Foster Grandparents dedicated a large amount 
of time to the Jumpstart program, children experienced an increased number 
of adults in the classroom for a substantial portion of the week. During the 
sessions, teachers reported that the volunteers used language and literacy prac- 
tices almost as often as they did. This suggests that the teachers positively 
viewed the volunteers’ contributions, though it is not possible to determine 
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whether this translated into more individualized adult-child interactions. Dur- 
ing CCT, although language and literacy instructional strategies were not used 
consistently across classrooms, teachers did report that volunteers engaged in 
a variety of one-on-one activities with children, providing individualized at- 
tention the children might not have received otherwise. 


Overall, these findings indicate that the components of the Jumpstart model were gener- 
ally delivered as intended, but limitations in the data make it challenging to assess to what extent 
the content and format of CCT was implemented, and whether the increased number of adults in 
the classroom consistently created more opportunities for high-quality, one-on-one interactions 
between children and adults. These implementation findings are discussed in greater detail in the 
remainder of this chapter. 


Volunteer Characteristics 


As described in the previous chapter, to become Foster Grandparents, seniors had to complete an 
application form and be willing to commit to 15 hours of services a week. Applicants were also 
interviewed by Jumpstart staff to gauge their interest and dedication to early childhood education. 
A total of 71 Foster Grandparents volunteered in the 11 participating study centers in the 2017- 
2018 academic year. The average age of the Foster Grandparents during the program year was 
74 years old. As shown in Table 2.1, most volunteers were women (87 percent) and most of them 
identified as either Asian (38 percent) or black (49 percent). ! 


The Foster Grandparents differed demographically from the children they served. Chil- 
dren in the centers served by Jumpstart across Los Angeles were predominately Latino, with a 
much lower percentage who identified as Asian or black. * Furthermore, many children were 
identified as being dual language learners and were from a Spanish-speaking home. Given these 
differences, Jumpstart offered training sessions that provided strategies for volunteers from dif- 
ferent backgrounds to teach and connect with dual language learners. 


Volunteer Preparation and Training 


The Foster Grandparents were expected to complete a series of pre-service training sessions in 
the months of August and September to prepare for the upcoming academic year. Of the 30 train- 
ing topics covered during pre-service training, Jumpstart identified 19 that were considered 


‘Information on volunteer characteristics is from the volunteer survey administered by Jumpstart at the be- 
ginning of the year. 

In 2017-2018, children served by the Jumpstart Foster Grandparent program in Los Angeles had the fol- 
lowing characteristics: About 96 percent were 3 or 4 years old (the remainder were 2 or 5 years old or older); 75 
percent were Latino and 15 percent were black; 52 percent were boys (48 percent girls); 82 percent were dual 
language learners; 59 percent were most comfortable speaking English; and 38 percent were most comfortable 
with Spanish. The children in the 11 study centers were similar on average to the children served by Jumpstart 
across the city. 
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Table 2.1 


Jumpstart Volunteer Characteristics 


Characteristic Percentage 
Age (years) 73.7 
Female 87.0 
Race 
Asian 38.2 
Black 48.5 
Other 13.2 


Highest education level 


Some high school or high school graduate 19.4 
Some college 26.9 
Associate's degree/certification 14.9 
Bachelor's degree 25.4 
Graduate degree 13.4 


Employment status 


Part time/full time 10.6 
Retired 74.2 
Other 15.2 
Primary language (English) 81.2 
Previous work experience with children 94.3 
Infants (3 months to 3 yrs) 16.7 
Preschool (3-5 yrs) 39.4 
Older youth (6-10 yrs) 6.1 
Non-specfified 56.1 
Sample size 71 


SOURCE: MDRC calculations based on Jumpstart volunteer pre-surveys distributed and collected in fall 
2017. 


NOTES: Findings in this table are based on data for 71 volunteer respondents, representing a 100 percent 
response rate. Sample size for individual items varies due to nonresponse or skipped questions. 
Responses may add to more than 100 percent as volunteers "checked all that applied" for certain 
questions. 
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“essential” for volunteers to complete prior to entering the classroom. On average, even though 
these training sessions required a significant time commitment, Foster Grandparents attended 18 
of the 19 essential sessions.* Furthermore, only two volunteers in the study started but did not 
complete the pre-service sessions, choosing to take a leave of absence. 


The Foster Grandparents also consistently attended the monthly in-service training pro- 
vided during the year. The average volunteer attended almost all (95 percent) of the sessions.* 
This sustained level of attendance is noteworthy given the extensive amount of time required for 
monthly training. The Foster Grandparents were not only willing to put in the time for pre-service 
training, but they were also very committed to their ongoing professional development and were 
able to meet Jumpstart’s expectations for participation. 


The Foster Grandparents also reported that the training was useful and prepared them to 
work in the classroom. Of the 32 Foster Grandparents who completed Jumpstart’s post-service 
survey, almost all (97 percent) agreed or strongly agreed that the Jumpstart training offered ef- 
fective preparation and guidance for service and program implementation. In addition, Foster 
Grandparents interviewed by the study team overwhelmingly praised the training as very helpful 
preparation for their work. Many of these volunteers made positive remarks about the structure 
of the training sessions, the supportive environment created by the facilitators, and the breadth of 
information they received: 


‘T’m going to tell you the training was tremendously a big help and it was 
really well done.... they did a demonstration on every aspect of the sessions 
with what to do, how to do it, what the purpose of that session is — the vocab- 
ulary words were stressed.” 


Center-Based Volunteer Teams 


As intended, each study center was supported by a team of four to seven Foster Grandparents, 
and most of these volunteers stayed on for the entire year. Of the 71 volunteers who joined the 
program in the study centers, only 11 (15 percent) left during the program year. In fact, the 85 
percent Foster Grandparent retention rate in this study is higher than that reported for Jumpstart’ s 
College Corps, exhibiting the commitment displayed by older volunteers.* The 11 Foster Grand- 
parents in the study who left did so early in the year, all during the first three months. No team 
had fewer than four members and only two center teams lost more than one volunteer during the 
program year. Furthermore, Foster Grandparents’ commitment to Jumpstart often extended be- 
yond just one program year. Of the 71 volunteers who participated in the program, 69 were re- 
turning participants, meaning the overwhelming majority had dedicated multiple years to the pro- 


gram. 


>These findings are based on sign-in sheets from the training sessions. 
4These findings are based on the timesheets of 16 randomly sampled volunteers. 
In 2017-2018, 75 percent of College Corps volunteers completed a full term of service. 
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Importantly, the Foster Grandparents also maintained a consistent presence at their cen- 
ters. The Jumpstart Foster Grandparent model had high expectations for volunteers to commit to 
serving at least three days a week for a total of at least 15 hours. On average, all volunteers whose 
timesheets were analyzed met this high threshold.° 


While on-site, Foster Grandparents were supported by the Jumpstart site manager, who 
periodically observed them and provided feedback. The number of observation and feedback ses- 
sions provided by site managers in the study varied by center. While all teams were observed at 
least twice, several received less than the five sessions recommended per year.? However, the 
volunteers had positive perceptions of the support received from site managers during these ses- 
sions. For instance, in the post-service survey, 97 percent of volunteers agreed or strongly agreed 
that observation and feedback from Jumpstart staff helped them focus on relevant domains and 
skills and engage more effectively with children. Most volunteer interviews also highlighted the 
sustained support they received from Jumpstart staff and teachers throughout the year: 


“They re very supportive. The teachers and the site manager, they’re very sup- 
portive because they can see what you are doing or working hard to help the 
children.” 


There does appear to have been some variation across centers with respect to volunteers’ 
perceptions of their fellow team members and team cohesiveness. A few volunteers described 
divisions within teams and an unwillingness to work together. Nonetheless, some volunteers did 
specify that their teams never let disagreements affect their work with the children. 


Jumpstart Sessions 


Overall, the Foster Grandparents successfully implemented the Jumpstart sessions as intended. 
As explained in Chapter 1, Jumpstart sessions should occur twice a week for 20 weeks, culminat- 
ing in a total of 40 sessions. In almost every Jumpstart study classroom, the expected 40 sessions 
were implemented, with one center offering 39 sessions. Additionally, every session had a se- 
quence of six structural components or segments of activities that Foster Grandparents were ex- 
pected to implement (see Table 1.1). Site managers observed volunteers implementing the six 
session components in addition to general classroom management strategies in the Jumpstart 
classroom.* In both the fall and spring, over 90 percent of classrooms implemented all of these 
components (see Table 2.2). This suggests that the Foster Grandparents offered the sessions reg- 
ularly and consistently, and that their administration of structural session elements was well 
aligned with the model. 


These findings are based on the timesheets of 16 randomly sampled volunteers. 


’These findings are based on a list of dates when Observation and Feedback sessions took place at each 
center, which was provided by Jumpstart. 


’These findings are based on data from the Session and Team Planning Checklist, an observational tool that 
site managers use to assess implementation during the sessions. 
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Table 2.2 


Percentage of Classrooms Where Foster Grandparents 
Implemented Specific Jumpstart Session Components 


Jumpstart Session Components 


Welcome 


Consistent meeting space for small groups was arranged and communicated 


Name cards were utilized 


Reading 
Volunteers worked with assigned small groups 
Volunteers used core storybook from session plan 
Read for Enjoyment or Read to Reconstruct strategy was used 
Reading included active effort to build children's vocabulary 


Circle Time 
Materials were prepared and organized 
There was an appropriate display and use of materials 
Activites were done as outlined in session plan (no additional activities) 


Introduction of Center Time 
There was a display and demonstration of center time activites 
Introduction included full explanations, without starting discussions 


Let's Find Out About It 
Materials were prepared and organized 
Team leader provided clear, accurate information to aid understanding 
Team leader used rich language throughout discussion 


Center Time 
Correct center time activities were included 
Adequate materials were available 
Adult was available to support each center 


Sharing and Goodbye 
Team leader asked children to share opinions and/or work at closing 
Information about next session was shared 
Team leader encouraged children to sing or chant goodbye song 


Classroom Management 
Volunteers verbally communicated session routine 


Volunteers used positive strategies to set limits and articulate expectations 


Classrooms that implemented all components 


Fall Percentage 


of Classrooms 


100.0 
90.9 


100.0 
100.0 
91.7 
91.7 


91.7 
100.0 
90.9 


80.0 
100.0 


100.0 
72.7 
83.3 


100.0 
100.0 
91.7 


100.0 
100.0 
100.0 


83.3 
70.0 


92.4 


Spring Percentage 


of Classrooms 


100.0 
100.0 


100.0 
100.0 
100.0 
100.0 


90.9 
100.0 
83.3 


100.0 
100.0 


91.7 
83.3 
100.0 


100.0 
100.0 
100.0 


100.0 
90.9 
100.0 


100.0 
75.0 


96.1 


SOURCE: MDRC calcuations based on Jumpstart session checklists completed by site managers in fall 2017 and spring 


2018. 


NOTES: Findings in this table are based on data for 12 Jumpstart group classrooms. Sample size for individual items varies 


due to nonresponse or skipped questions. 
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Throughout the Jumpstart sessions, Foster Grandparents were also expected to use in- 
structional practices to promote children’s language and literacy development in three key do- 
mains: oral language, book and print knowledge, and phonological awareness. Information on 
volunteers’ use of these practices was obtained from a survey administered to the lead teachers in 
the Jumpstart classrooms. Teachers were asked to rate the frequency with which the volunteers 
used 10 instructional practices that one would expect to see during the sessions (such as creating 
opportunities for children to interact in one-on-one conversations, giving age-appropriate defini- 
tions of words, and helping children connect letters and sounds). As explained in Chapter 1, many 
of these items were adapted from the Early Language and Literacy Classroom Observation 
(ELLCO) Pre-K Tool, a well-established observational tool designed to capture the quality of 
language and literacy instruction in preschool classrooms.’ 


As shown in Table 2.3, teachers reported that they did observe Foster Grandparents using 
these language and literacy practices during the sessions. One practice in particular — “used 
songs, chants, nursery rhymes, and poems” — was observed at a high frequency (often or almost 
always) in almost all the classrooms (92 percent). Other practices were observed at a high fre- 
quency in around 50 percent of Jumpstart classrooms, such as “initiated conversations that max- 
imized talk and built on oral language skills” or “played games that got children to listen for 
beginning and ending sounds.” Findings in Table 2.3 also indicate that certain strategies may have 
been easier than others to integrate into the Jumpstart sessions. For example, while volunteers in 
73 percent of classrooms frequently “created opportunities for children to interact in small group 
conversations,” volunteers “initiated conversations that maximized talk and built on oral language 
skills” at a high frequency in only 50 percent of classrooms. While volunteers frequently “used 
songs, chants, nursery rhymes, and poems” in 92 percent of classrooms, they only frequently 
“used song and word games to isolate sounds in words” in 50 percent of classrooms. This suggests 
that some practices that target more specific language and literacy skills may require more in- 
structional experience to be consistently employed. 


Another important factor in making sure children benefit from the sessions is attendance. 
The more sessions children attend, the more opportunities they have to interact with the volun- 
teers. Jumpstart’s goal is that children attend at least 32 of the 40 sessions offered. In 5 of the 11 
study centers, less than 80 percent of children in the Jumpstart classroom met that benchmark due 
to mobility, attrition, sporadic school attendance, and other factors outside of Jumpstart’s control. 


Child-Centered Time 


As discussed in Chapter 1, CCT happens outside of the Jumpstart session during regular class 
time. The goal of CCT is for the volunteers to engage with and provide their partner children with 
individualized support, focusing on their language and literacy skills and their social-emotional 
development. All children in the Jumpstart classrooms should receive CCT for at least three hours 
a week; and a subset of children in the CCT-only classrooms receive it as well. In addition to 


*Polk (2013). 
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Table 2.3 


Percentage of Jumpstart Classrooms with High-Quality Language 
and Literacy Instruction Frequently Provided During Sessions 


Practices 


Created opportunities for children to interact in 


one-on-one conversations 


Created opportunities for children to interact in 


small group conversations 


Initiated conversations that maximized talk and built 


on oral language skills 


Asked open-ended questions that would encourage 
children to talk about their thinking 


Talked to children using words slightly beyond their 


understanding to broaden knowledge of words 
Gave age-appropriate definitions of words 
Used songs, chants, nursery rhymes, and poems 


Played games that got children to listen for 


beginning and ending sounds 


Used expressions, tone of voice, gestures, or references 


to illustrations to support understanding during reading 
Used song and word games to isolate sounds in words 


Helped children connect letters and sounds 


Sample size 


Percentage* 


41.7 


72.7 


50.0 


58.3 


58.3 


66.7 
91.7 


41.7 


58.3 


50.0 


58.3 


12 


SOURCE: MDRC calculations based on teacher surveys distributed and collected in spring 


2018. 


NOTES: Findings in this table are based on data for 12 Jumpstart group teachers. Sample size 


for individual items varies due to nonresponse or skipped questions. 


All items use a 5-point frequency scale: 1 (never), 2 (seldom), 3 (sometimes), 4 (often), and 
5 (almost always). An item was considered to be implemented frequently (high quality) if 


teachers responded with either a 4 (often) or 5 (almost always). 


*Percentage includes the classrooms whose lead teachers responded to the item with 


eithera4ora5. 
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benefitting volunteers’ partner children, CCT is also expected to improve outcomes at the class- 
room level because it increases the number of adults in the classroom. That creates more oppor- 
tunities for adults and children to interact one-on-one to support foundational language skills. 


In the study centers, the Foster Grandparents spent more time providing CCT than the 
four to five hours that was expected of them. On average, volunteers logged 14 hours of CCT a 
week, demonstrating their commitment to the program and the children.'° The volunteers who 
split their time between two classrooms (Jumpstart and CCT-only) reported spending an average 
of four and a half hours of CCT in the Jumpstart classroom each week. Thus, children in the 
Jumpstart classroom generally received a minimum of three hours of CCT a week, as intended 
by the model.!! 


The time spent during CCT is unstructured by design so that the volunteers can tailor 
their support based on whatever activities are happening in the classroom on a given day. Still, 
the expectation is that the volunteers will work with their assigned partner children on language 
and literacy development. Given the available data, however, it is challenging to assess precisely 
how much one-on-one time each child received from the volunteers during nonsession time. '? In 
interviews, volunteers described working with other children in the classroom in addition to work- 
ing with their partner children. This suggests that it may have been challenging for the Foster 
Grandparents to provide one-on-one time to their partner children and give them their undivided 
attention, because all children may have seen them as “grandparents” and wanted to engage with 
them. 


One source of information about volunteers’ practices during CCT comes from the 
teacher survey. Lead teachers in the study centers were asked to report on whether the volunteers 
were ever observed using language and literacy instructional strategies during one-on-one activ- 
ities in CCT.!? Foster Grandparents in all classrooms were observed using at least one of the 
recommended strategies. On average, teachers reported that the Foster Grandparents used these 
language and literacy strategies during one-on-one CCT in 35 to 70 percent of classrooms de- 
pending on the strategy (see Table 2.4). For this evaluation, Jumpstart had identified a goal of 
having volunteers use five or more strategies during one-on-one CCT; this happened in a little 
over half (55 percent) of the study classrooms. * 


°These findings are based on the timesheets of 16 randomly sampled volunteers. 

'There were three volunteers who logged an average of slightly under three hours of CCT a week in the 
Jumpstart classroom. 

"Tn their timesheets, volunteers track the amount of time spent in classrooms outside of the Jumpstart ses- 
sions but not the number of hours they specifically spend with their partner children during nonsession time. 
3MDRC and Jumpstart worked in partnership to identify certain key language and literacy strategies that 
Foster Grandparents could potentially use to engage children one-on-one during CCT. 


‘Additionally, the survey did not ask about the frequency at which these strategies were employed, so it is 
not possible to determine what portion of CCT was devoted to employing the recommended language and liter- 
acy instructional strategies. 
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Table 2.4 


Percentage of Classrooms Where Foster Grandparents Employed 
Specific Language and Literacy Strategies during One-on-One CCT 


Strategies 

Directly taught new words to children 

Connected a new word to a word children already know 
Had children repeat new words 

Asked comprehension questions about a story 

Helped children say sounds in unfamiliar words 
Embedded use of new words in children's daily experiences 
Noted specific features of print and letters 

Asked children where to start reading 

Counted words in text 

Pointed out print within pictures 


Classrooms with 50% or more of the 


strategies employed by Foster Grandparents* 


Sample size 


Percentage 
55.0 
55.0 
55.0 
70.0 
60.0 
45.0 
60.0 
40.0 
35.0 


60.0 


55.0 


20 


SOURCE: MDRC calculations based on teacher surveys distributed and collected in spring 2018. 


NOTES: Findings in this table are based on data for 20 teachers (Jumpstart and CCT-only) who had 
Foster Grandparents assigned to their classrooms. Sample size for individual items varies due to 


nonresponse or skipped questions. 


*Jumpstart considers a classroom as having implemented CCT-time with high quality if 50 


percent or more of the strategies were implemented by the Foster Grandparents. 


Overall, the Foster Grandparents did devote many hours to CCT, more than expected, but it is not 
possible to determine the amount of time volunteers specifically devoted to working one-on-one 
with their designated partner children during that time. Based on teacher reports, it appears that 
the Foster Grandparents used language and literacy instructional strategies during one-on-one 
nonsession time, though not as consistently as intended by the Jumpstart model. It is worth noting 
that the volunteers could have been employing other practices that were not captured, such as 


strategies focused on social-emotional development. 
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Service Contrast 


An important driver of Jumpstart’s effect on children’s development is the extent to which the 
presence of the volunteers substantially changed the learning environment compared to what chil- 
dren would have experienced otherwise. The bigger the “service contrast” or service differential 
created by the volunteers, the bigger the effect one might expect the model to have on children’s 
development. !° In the study centers, the service contrast could be explored by considering whether 
and how the experience of children in the classrooms served by the volunteers differed from the 
experience of children in the business as usual classrooms. 


As discussed in Chapter 1, the Jumpstart model primarily intends to enhance the learning 
environment by increasing the number of adults in the classroom, so that children will have op- 
portunities for individualized attention that teachers might otherwise not have the capacity to pro- 
vide. In the study centers, the presence of the volunteers did substantially increase the number of 
adults in the classroom for a large portion of the week. Each center was supported by a team of 
four to seven Foster Grandparents, most of whom stayed for the full year. Volunteers spent an 
average of 3.7 days in center classrooms, logging an average of 22 hours a week. During the 20 
weeks of Jumpstart session implementation, only two volunteers were absent for more than three 
weeks. !° 


It is not possible to determine whether the increased number of adults in the classroom 
led to more one-on-one attention for children during Jumpstart session time. However, the lead 
teacher survey did provide information on the extent to which the volunteers used different lan- 
guage and literacy practices, including those that one would expect to see more often in an envi- 
ronment where instruction is more individualized (for example, initiating conversations with chil- 
dren that maximize talk). Teachers were asked to report on volunteers’ use of language and 
literacy practices in the classroom during session time, as well as their own use of these practices. 
Teachers’ reports of their own instructional practices may have overrepresented the extent to 
which they actually used them. However, comparing teachers’ self-ratings against their ratings of 
the Jumpstart volunteers’ practices provides useful information about how trained and experi- 
enced educators (lead teachers) viewed the volunteers’ role in the classroom during the Jumpstart 
sessions. 


The findings suggest that during sessions, teachers perceived the volunteers as contrib- 
uting to language and literacy instruction almost at a similar level as a trained classroom teacher. 
As for their own instruction, the lead teachers consistently self-reported that they used key lan- 
guage and literacy instructional practices “often” or “almost always,” suggesting that teachers in 
the study centers viewed themselves as consistently engaging children with these practices out- 
side of session time.!’ This was true across all three classroom types (Jumpstart, CCT-only, and 


‘The service contrast findings in this section are for all 11 study centers. Please refer to Appendix D for the 
service contrast findings for the five centers in the impact study. 


‘These findings are based on the timesheets of 16 randomly sampled volunteers. 


Frequency of use was calculated using the mean of a five-point scale: 1 (never), 2 (seldom), 3 (sometimes), 
4 (often), and 5 (almost always). Teachers in Jumpstart classrooms also reported spending an average of 4.4 days 
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business as usual). Teachers also reported that during the Jumpstart sessions, the volunteers used 
key language and literacy instructional practices at slightly lower but similar frequencies as they 
themselves did (Table 2.5). This, in turn, suggests that the presence of additional trained volun- 
teers in the center classrooms during sessions may have potentially created more opportunities 
for positive one-on-one interactions between children and adults. However, given the available 
data, it is not possible to determine whether these opportunities translated into more individual- 
ized attention for children in the Jumpstart sessions. 


The time Foster Grandparents spent in center classrooms was not limited to leading 
Jumpstart sessions. They also spent many hours focused on CCT in both the Jumpstart and CCT- 
only classrooms, which is specifically designed for volunteers to provide individualized attention 
to children during teacher-led instruction time. Teachers were asked to report on the kinds of one- 
on-one activities that Foster Grandparents engaged in with children during CCT. As seen in Table 
2.6, teachers observed volunteers using each one-on-one activity in 45 percent or more of class- 
rooms. Though teachers were not asked to self-report on how often they engaged children in one- 
on-one activities during class time, the activities during CCT occurred in addition to regular 
teacher-led instruction. This suggests that the addition of Foster Grandparents in the classroom 
during CCT did result in more one-on-one interactions between children and adults. Still, the 
survey did not ask how often the one-on-one interactions occurred, so it is not possible to deter- 
mine what proportion of CCT was spent focused on individual interactions. Additionally, as de- 
scribed in the CCT section above, the use of focused language and literacy strategies during the 
additional one-on-one time may not always have been as consistent as intended by the Jumpstart 
model. 


Overall, Foster Grandparents’ presence during Jumpstart sessions and CCT contributed 
to an increased number of adults in the classroom. During sessions, Foster Grandparents offered 
language and literacy instruction generally comparable to that of the teachers, but it is not possible 
to determine if their instructional practices led to more individualized interactions between chil- 
dren and adults. During CCT, volunteers were observed engaging children in one-on-one activi- 
ties (though the frequency of those activities was not measured) and employing language and 
literacy strategies during that one-on-one time (although strategies were not always used as con- 
sistently as intended by the Jumpstart model). Nonetheless, volunteers did take advantage of CCT 
to interact with children in their classrooms one-on-one, allowing these children to experience 
more individualized interactions than children in business as usual classrooms. 


a week on language and literacy instruction outside of the Jumpstart sessions. In several of the study centers, 
teachers used a curriculum to support language and literacy and the social-emotional development of children, 
which may explain this pattern of results. This included the DLM Early Childhood Express curriculum, which 
provides additional STEM resources and materials, and High Scope, a comprehensive curriculum that includes 
language and literacy and social-emotional learning. 
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Table 2.5 


Comparing Teacher and Foster Grandparent Use of Language 
and Literacy Instructional Practices in Jumpstart Classrooms 


Foster 
Practices (5-point scale)* Grandparent Teacher 
Created opportunities for children to interact in 3.6 4.3 
one-on-one conversations 
Created opportunities for children to interact in 4.2 4.2 
small group conversations 
Initiated conversations that maximized talk and built 3.4 4.2 
on oral language skills 
Asked open-ended questions that would encourage 3.6 4.2 
children to talk about their thinking 
Talked to children using words slightly beyond their 3.9 4.1 
understanding to broaden knowledge of words 
Gave age-appropriate definitions of words 3.8 4.3 
Used songs, chants, nursery rhymes, and poems 4.3 44 
Played games that got children to listen for 3.7 3.9 
beginning and ending sounds 
Used expressions, tone of voice, gestures, or references 3.8 4.3 
to illustrations to support understanding during reading 
Used song and word games to isolate sounds in words 3.8 4.3 
Helped children connect letters and sounds 3.8 4.2 


SOURCE: MDRC calculations based on teacher surveys distributed and collected in spring 2018. 


NOTES: Findings in this table are based on data for 12 Jumpstart group teachers. Sample size for 
individual items varies due to nonresponse or skipped questions. 

aThe values presented in this table are calculated from the mean of a 5-point scale: 1 (never), 2 
(seldom), 3 (sometimes), 4 (often), and 5 (almost always). 
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Table 2.6 


Percentage of Classrooms Where Foster Grandparents Employed 
Specific One-on-One Activities during CCT 


Activites Percentage 
Read a book 70.0 
Practiced recognizing letters 80.0 
Practiced recognizing numbers 45.0 
Completed arts and crafts projects 50.0 
Sang songs or made music 60.0 
Used building blocks 55.0 
Played dress-up 50.0 
Played with dolls or toys 55.0 
Supported children during transitions 75.0 
Engaged children during snack or meals 75.0 
Sample size 20 


SOURCE: MDRC calculations based on teacher surveys distributed and collected in spring 2018. 


NOTES: Findings in this table are based on data for 12 Jumpstart group teachers and 8 CCT-only group 
teachers. Sample size for individual items varies due to nonresponse or skipped questions. 
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Chapter 3 


Impacts on Children’s Language and Literacy and Social- 
Emotional Development 


This chapter examines whether the Jumpstart Foster Grandparent (JFG) model shows promise 
for improving children’s language and literacy development and their social-emotional outcomes. 
As explained in Chapter 1, this analysis is based on 5 of the 11 study centers. In each one, newly 
enrolled children were randomly assigned to one of three types of classrooms: a Jumpstart class- 
room, where the full Jumpstart model was provided by the volunteers (sessions and child-centered 
time or CCT); a CCT-only classroom, where some children were partnered with a volunteer to 
receive one-on-one attention; and a business as usual classroom, where the volunteers did not 
provide services. The impact of the full Jumpstart model can be estimated by comparing the 
spring outcomes of children in the Jumpstart and business as usual classrooms. Similarly, the 
classroom-level impact of CCT can be estimated by comparing the spring outcomes of children 
in the CCT-only and business as usual classrooms. 


As discussed in Chapter 1, the findings from this analysis should be considered explora- 
tory and not used to make definitive conclusions about the Jumpstart model’s effect, for two rea- 
sons. First, although children were randomly assigned to classrooms, teachers were not, so the 
estimated effects reported in this chapter could be confounded with differences in teacher quality 
across the three types of classrooms. (As will be discussed in this chapter, the pattern of findings 
suggests that this might be the case.) Second, because the analysis is based on a small number of 
children, the margin of error is large. In practice, this means that the effects of the Jumpstart model 
would have to be very large — greater than those of most educational interventions — to be able 
to conclude that they are statistically significant. However, even though the findings are unlikely 
to be statistically significant, the potential for positive effects can still be explored by examining 
their magnitude and whether they are going in the right direction. 


The overall finding from the analysis is that the children in the Jumpstart and CCT-only 
classrooms did not make greater gains than children in the business as usual classrooms. Children 
in all three types of classrooms (Jumpstart, CCT-only, and business as usual) made similar gains 
during the year on their language and literacy skills and their social-emotional competencies. The 
remainder of this chapter provides further information on the characteristics of the children in the 
analysis sample and a more detailed discussion of the impact findings. ! 


‘Because of the small sample size in the impact study, the research team conducted an additional exploratory 
analysis to bolster the analytic sample. In this exploratory analysis, children in Jumpstart and CCT-only class- 
rooms in study centers where random assignment did not happen were matched to similar children in business 
as usual classrooms. This combined study sample of children was added to the sample of randomized children. 
The pattern of findings based on this larger sample are similar to those from the randomized experiment and also 
inconclusive. (See Appendix E for details). 
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Baseline Characteristics of Children 


As explained in Chapter 1, children’s fall and spring outcomes in this study were measured using 
the Desired Results Developmental Profile — Preschool. The DRDP-PS is an observational tool 
used by early childhood education teachers in California to rate children’s development in differ- 
ent domains. The key outcomes of interest for this study were scores in three domains that are 
best aligned with Jumpstart’s theory of change: English Language and Literacy Development, 
Interpersonal Skills, and Self-Regulation. Each domain score was based on several items, and a 
partial credit Rasch model was used to create composite scores as recommended by the DRDP 
developers. However, the scale of these scores does not have an interpretable metric, so in this 
chapter, children’s average scores are discussed based on the nine-point scale that teachers use 
when rating children. In addition, differences in child outcomes between classrooms (Jumpstart, 
CCT-only, and business as usual) were scaled as an “effect size,” a metric that is widely used for 
gauging whether the magnitude of a program’s effect is large or small. It is defined as the be- 
tween-group difference for an outcome expressed as a proportion of the standard deviation for 
that outcome. 


Table 3.1 shows the fall DRDP-PS scores and other characteristics of children in the 
Jumpstart classrooms and the business as usual classrooms in the analysis sample. Before the 
Jumpstart program started, children in the Jumpstart classroom had an average English language 
and literacy score of 5.3 and a social-emotional score of 6.4 points on a 9-point scale. Importantly, 
children in the Jumpstart classrooms and the business as usual classrooms had similar scores at 
baseline (the latter group had average scores of 5.2 and 6.3 on the two main outcomes). On aver- 
age, children in the Jumpstart and business as usual classrooms were also similar with respect to 
their age (3 or 4 years old), gender (about 40 percent were boys), race/ethnicity (about 75 percent 
of children were Hispanic), and language (about 45 percent of children had English as their sec- 
ond language). Many of the differences between the groups’ baseline characteristics, including 
children’s scores on the fall DRDP-PS, were small in magnitude (less than 0.25 as an effect size).? 
This indicates that random assignment was successful at creating groups of children with similar 
outcomes and characteristics at baseline. Children in the CCT-only classrooms and the business 
as usual classrooms were also similar with respect to many baseline characteristics, although there 
is a Statistically significant difference in children’s baseline social-emotional development scores 
(Table 3.2). 


Effect of the Jumpstart Foster Grandparent Model 


Table 3.3 presents the average spring DRDP scores of children in the Jumpstart classrooms and 
the business as usual classrooms. As shown in this table, children in the Jumpstart classroom had 


Effect sizes are based on the standard deviation of the Rasch-scaled version of the DRDP score, for the 
pooled analysis sample. 


>The What Works Clearinghouse considers that baseline equivalence is achieved when differences are less 
than 0.25 on baseline measures of the outcome. 
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a spring DRDP score of about 6 points on their English language and literacy development, while 
children in the business as usual classroom had a score of 6.5 points. Thus, the estimated effect 
of the full Jumpstart model on children’s English language and literacy scores was numerically 
negative, but small in magnitude and not statistically significant (effect size = -0.24, p-value = 
0.133). For social-emotional development, the average spring score of both groups of children 
was 7.7. Thus, the estimated effect on social-emotional scores was numerically positive but small 
in size and not statistically significant (effect size = 0.01, p-value of 0.951). 


Although these estimated effects are small, it is important to note that children still made 
positive gains during the year. As discussed above, in fall 2017, children in the Jumpstart class- 
rooms had an average English language and literacy score of 5.3 and a social-emotional score of 
6.4 on a 9-point scale. When children were assessed again in spring 2018, their language and 
literacy scores increased, on average, by 0.7 points, and their social-emotional scores increased, 
on average, by 1.3 points. Children in the business as usual classrooms made similar gains, which 
suggests that the Jumpstart model may not have improved children’s scores beyond the gains that 
children would have made otherwise.* 


Table 3.4 presents the average spring DRDP scores of children in the CCT-only class- 
rooms and the business as usual classrooms. The estimated classroom-level effect of CCT on 
children’s English language and literacy scores was positive and moderate in magnitude, but not 
statistically significant (effect size = 0.11, p-value = 0.658). The estimated effect on the social- 
emotional development score was also positive and moderate in magnitude but not statistically 
significant (effect size = 0.32, p-value = 0.127). Children in the CCT-only classrooms made nu- 
merically larger (though not statistically significant) gains during the year than children in the 
business as usual classrooms on these outcomes.°* 


This pattern of findings suggests that the estimated effect of providing CCT-only was 
numerically larger than the effect of providing the full Jumpstart model. However, the reverse 
should have been true and the effect of the full Jumpstart model should have been larger than the 
effect of classroom-level CCT: In Jumpstart classrooms, all children received sessions and all 
children received CCT (full Jumpstart model); in CCT-only classrooms, only a subset of children 
received CCT. 


This unlikely pattern of results suggests that there may have been differences in teachers 
across classrooms, and that these differences may have been confounded with the effect of 
Jumpstart. As discussed in Chapter 1, generally across classrooms, teachers were highly experi- 
enced educators, with an average of 14 years or more of early education teaching experience, and 


‘Tn the fall, the children had an average score of 5.2 on the English language and literacy development scale 
and 6.3 on the social-emotional development scale. By the spring their language and literacy scores had in- 
creased, on average, by 1.3 points, and their social-emotional scores had increased, on average, by 1.4 points. 

>The average English language and literacy and social-emotional scores of children in CCT-only classrooms 
in the fall were 5.0 and 5.8, respectively. By the spring, their language and literacy scores increased, on average, 
by 1.5 points, and their social-emotional scores increased, on average, by 2.2 points. 


42 


‘ajeos je}UsWUdOJaAep jUIOd-suIU |eUIBIUO ay} UO peseq ‘UMOYUS 
osje aie SWS}! JUBAZ/SJ BY} SSOUDe B1ONS BHesJaAe 9|dUWIS S,USJpIIYO JO} Sues dnoJB ‘uoNe}esdsajuU! |NJBuIUesW e SAeY JOU OP SaJONS | Yy] BsNeoeg 
‘jusosed OL =, pue 
‘\uddJ0d G=,, ‘JUSDOd | =,,4 ‘SMO||OJ SE S|BAB] SOUBDIJIUBIS YM SOUSIOYIP Pa}eLUNSS YOeS Jo soUedIIUBIS |edISI}e}S BU} SSasse 0} pasn sem }S9}-} 
“SW9}1 BS9Y} UO pasegq a/edS d}ISOdWOD e& 9}ea19 0} paSn SEM japOW | Yj] }pes0-jeed 
V ‘(SIDS [euossediajyu! Buunseow Sud}! SAY pue UO!}E/NBal-jjas Bulnseaw SW}! INOJ) SW}! BUIU SEPNjoU! ajeos JUSLUdO|SANq |BUOI}OW F-[e1ID0S BY fq 
“SWAd}I BS9y} UO paseq ajeds a}Isodwiod e a}ea10 
0} pesn sem japow [HY] }peso-jensed vy ‘jUsuUdojeAsp ADes9}!| pue eHenBue| s,usspjiyo Hulnseaw Swe}! O| UO peseq si! ajeds siy} ‘ysi|]Gbuy sem ebenbue| 
Aseulid BSOUM UdJIPIYO JO4 "}JUsLUdOJeAep eHenbuel-ysijbuy s,usipyiyo Bulnsesw Sus}! JNO} UO paseq SI ajeds siy} ‘siguse9] BBenbHue] ysij|6uy Jo4e 
"SJOBJUN[OA Pe}SAWINC BU} WO SPDIAJOS BAIBN9J JOU PIP yey} WOOUSSe|O 
e 0} poubisse 3j9m OYM UsUp|lyo peubisse AjWopues JO} SueawW pejsn[pe-uolssasHeJ au} ae UWWNIOD }x9U By} Ul SANjeA ,dnoig jensp se sseulsng,, 
SUL ‘JEpowW AjUO- 19D BY} PaAleoei yey} WOOUSSe]O e 0} pouBbisse AjWOpUes B1aM OYM UdJPIIYO JOJ SUBS PEAJaSqo 9y} ae ,dNoId AJUO-1LD9,, pe|eqe| 
ULUN|OD 9y} Ul SaNjeA aU “SuedA BAI }SE] OY} Ul POAOW pey AjilwWe} BY} SOLU} JO JOqLUNU Bu} pUe ‘SdWe}s poo} paeAledal AjIWeJ S,pIIyO eB JBYJOUM ‘azis AjlWe} 
‘uolyeonps jusied ‘awoy ye ueyods ebenbue| ‘abe say) ‘sepueb sau} ‘Ue|d UOMeONpsS pszijenplAipul ue pey PIYO dy} JOYJOYM ‘JauUJee] eHenbHue] jenp e 
SEM PIIYO BY} JOYJOUM ‘S]UBLUSSESSe Sq-dQNC 9} UO $asods /|.0Z |e} S,USIPII4O 0} JOSdse1 YyyM dnosB uosedwos pue AjUC-1DD Buy} UBEMJEq SBOUSJAYIP 
Bulujewes Aue se |jam se ‘syoojq JuswUBIsse WopuUed JO} Bul|joujUO9 ‘Seuenbs }see] Aleuipso Bulsn paysn{[pe-uocisseibed ase sjoaye peyewnsy 
‘(dnou6 jensn se sseuisng) siaajunjoA 3u} Aq peyoddns jou aJam ey} SWIOOJSSe]O 0} pue (dnoB AjUO-1 DD) Susa}UNJOA BU} WO’ [DD Ajuo 
PSAIBN9J JEY} SWOOJSSEIO 0} pouBbisse AjWOPUes 819M OYM UdJP|IYD SepNjoul sjdwes sul “gLOz Bulsds pue /10Z [1e} Ul Sd-dGNd ey} Bulsn syayoes} wI9uU} 
Aq passesse 3Jam OUM pure Apnjs 9u} U! UOedIoNJed 1194} 0} pa}U9SUOD S}UsIed BSOYM UsJp|IUD UO peseq aie 9|qe} Siy} Ul peyode sasAjeue SUL ‘SALON 


*8L0Z Buuds ul (Sd-daya) [OOYDSSJd — 9|Old jejusudojereqg s}/NSeY psJIsoq 94} UO peseq sBunes Jayoes| -FOUNOS 


ee ov USJIP|IYO JO JEQUINN 
gL 08 (julod-g) a]e0s jejusUdojeAeq 
09€°0 zo 0 610 80°0 820 (2/298 1 YI) SIPS |euosiediaju| 
92 08 (julod-g) a]e0s jejusLUdOjeA9Gg 
L20°0 « Ovo Ge0 L0°0 Z€°0 (9/298 | YI) UOHeINBay-jeS 
ZL 0's (julod-g) a]e9s jejusUUdojeAeqg 
ZZ1°0 ce'0 62'0 20°0 9€'0 (2leos Ly]) Juswdojeasq jeuolow z-je1Is0g 
s‘9 s‘9 (julod-g) a]e9s jejusUUdojeA9q 
899'0 LL'O OL'0 OL'0 020 (2/298 1y|) jusudojersq Acesey1] pue eHenbue7 ysi|6ug 
soUuslEyIG peyewsy aZIS sOUBaHIG dnoig jensn dnoig @WW09}NO 
JO} anjeA-d yoy peyeulysy se ssouisng AlUO-LOD 


ajduieg yoeduy ‘saui093noO S,uaip|iyD UO (19D) aul, pasajUaD-p]lYD Jo J9aj4y payewNsy 
ve a1geL 


43 


all had at least some college education. However, teachers in CCT-only classrooms were slightly 
younger and more educated (for example, more likely to have a bachelor’s degree).° These dif- 
ferences, and other unmeasured differences, might affect the quality of language and literacy in- 
struction received by children across classrooms (alternatively, it could affect how these teachers 
rated children’s development on the DRDP). Therefore, some of the between-classroom differ- 
ences in Tables 3.3 and 3.4 may be due to differences in teachers across classrooms, as opposed 
to the effect of Jumpstart.’ 


Overall, the findings from this analysis suggest that children in the Jumpstart classrooms 
did make gains, but they were not greater than the gains made by children in the business as usual 
classrooms. To what extent this can be attributed to the Jumpstart model is difficult to determine 
given the study’s limitations. 


®See Appendix F, Table F.1 and F.2. 

Another potential factor affecting the findings is a possible ceiling effect on the DRDP scores. The spring 
scores of children in the analysis sample were clustered between 7 and 9 (on a 9-point scale), with a larger 
proportion of children at a score of 9. This suggests a possible ceiling effect that may have dampened Jumpstart’s 
potential to have an impact on children’s DRDP scores. 
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Chapter 4 


Conclusion 


The Jumpstart Foster Grandparent (JFG) program aimed to provide language and literacy instruc- 
tion and supports for preschool children from underserved communities using an intergenera- 
tional volunteer model. That mission required a significant investment in understanding commu- 
nity needs. It also called for a substantial amount of training to effectively prepare a corps of 
seniors to provide high-quality instruction and foster meaningful relationships with children. 


The findings from this study of the JFG model highlight several important lessons about 
the implementation of volunteer-based educational interventions and how to conduct evaluations 
of programs in early childhood education settings. These lessons are discussed in this chapter. 


After the time period covered by this study (2017-2018), Jumpstart made several changes 
to its model and training that are well aligned with this study’s recommendations; these changes 
are discussed at the end of this chapter. In summer 2019, Jumpstart also decided to discontinue 
its Foster Grandparent program, which represented about 4 percent of its total volunteer corps.! 
This shift will allow the organization to focus its resources on the College Corps, a group that is 
better aligned with Jumpstart’s goal of training volunteers who will go on to become part of the 
workforce as educators and teachers. Some of the lessons from this study may help inform the 
delivery of the Jumpstart model as the organization commits to delivering its model exclusively 
through the College Corps. 


Lessons About Implementing Volunteer-Based Intergenerational 
Programs 


e It is possible to build a corps of senior volunteers who will commit signif- 
icant amounts of time and energy to providing services to children. 


One of the most notable findings from this evaluation was the volunteers’ high level of 
commitment to the Jumpstart program. Even though many of the JFG volunteers faced transpor- 
tation, health, and social barriers, the Foster Grandparents attended many hours of pre-service and 
in-service trainings; they also met, and sometimes exceeded, the service requirement of 15 hours 
a week. In interviews, volunteers expressed appreciation for the JFG Program, describing it as a 
reason to get out of bed each week, a way to stay healthy, and a way to get joy from children 
while giving back to the community. The JFG program demonstrates that a committed corps of 
senior volunteers is possible, and that the volunteers derive significant personal benefits from 
volunteering. 


‘Jumpstart (2019). 
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e Some instructional practices are more challenging for volunteers to im- 
plement than others. 


Several instructional practices are considered by the field of early childhood education to 
be effective in supporting children’s language and literacy development. The findings from this 
study suggest that although the volunteers implemented the language and literacy practices they 
learned in training, the frequency with which those practices were used varied from center to 
center. During the Jumpstart sessions, the Foster Grandparents were more likely to use practices 
that were easier to implement, such as songs and chants, than ones that may have required more 
instruction, such as playing games that get children to listen for sounds and take turns during 
conversations. 


High-quality language and literacy practices are challenging even for experienced edu- 
cators to implement. Staff members who supervise volunteers should provide regular training to 
help reinforce and support the knowledge of these practices, and regularly emphasize the im- 
portance of employing them frequently. Such training could include workshops about ways to 
integrate several of the practices throughout a Jumpstart session. In addition to having program 
staff observe and provide feedback to the volunteers, early childhood education center staff (such 
as lead teachers, who spend the most time with the volunteers) could also be asked for their feed- 
back on a regular basis, whether informally or through a survey or checklist. Teachers’ responses 
could be used to help determine which instructional practices to focus on with volunteers during 
training sessions. 


e Karly childhood education teachers may be spending a significant amount 
of time on language and literacy development, which can have implica- 
tions for program delivery. 


The findings from this study indicate that teachers were “often” or “almost always” using 
practices that target language and literacy development. The frequency of language and literacy 
instruction reported by teachers in the study centers may reflect a growing push toward improving 
instructional quality in early childhood education settings. California specifically has recently 
demonstrated a commitment to improving the quality of preschool instruction in response to in- 
creased accountability.* The state’s investment in preschools increased by more than $364 million 
in 2018 (accounting for an increase of more than $1,000 per child), and the state has successfully 
met the recommended benchmarks for early learning and development standards.* This increased 
investment has enabled preschools (including some of the ones in this study) to invest in and 
adopt language and literacy curricula in their classrooms.’ This trend toward richer language and 
literacy instruction in preschools should be taken into account by organizations that offer 


Though California has struggled to meet the National Institute for Early Education Research’s (NIEER) 
quality benchmarks in the past, the most recent report identified them as a “State on the Move,” suggesting a 
strong push toward improvement (Friedman-Krauss et al., 2019). 

3Friedman-Krauss et al. (2019) 

4Curricula used in Jumpstart study centers included the DLM Early Childhood Express curriculum, which 
provides additional STEM resources and materials, and High Scope, a comprehensive curriculum that includes 
language and literacy and social-emotional learning. 
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supplemental services in early childhood education classrooms. For example, it may be important 
to consider how the supports provided by volunteers interact with (or supplement) the other cur- 
ricula used by teachers, to ensure that the language and literacy activities delivered by both are 
aligned across a child’s day and week. 


e High child mobility and sporadic attendance in early childhood education 
settings can make it challenging for staff members to build relationships 
with children, which highlights the programmatic importance of child- 
centered strategies. 


In this study, many children left the centers partway through the academic year and were 
not assessed in the spring. > Children with inconsistent or unstable school attendance in preschool 
miss out on instruction and supports that will prepare them for kindergarten. Child mobility also 
makes it more challenging for adults in early childhood education centers to build relationships 
with children. Organizations like Jumpstart that focus on relationships need to work especially 
hard to make sure that one-on-one time with children (like CCT) is maximized during the time 
children may be enrolled at the center, no matter how short. Studies have shown that some pre- 
school children receive little or no individualized attention from their teachers, so CCT also pro- 
vides an opportunity for volunteers to make a crucial contribution to children’s language and 
social-emotional development. ° 


e Volunteers balance general classroom support with spending one-on-one 
time with children, and they may need some structure for the time they 
spend in the classroom. 


Findings from this study indicate that it can be challenging for volunteers to provide in- 
dividual attention solely to their partner children in a classroom setting when other children are 
present and may need or want their attention, too. One strategy for helping the volunteers balance 
general classroom support with spending more time with partner children would be to have them 
engage in activities with their partner children for a dedicated amount of time, and to set specific 
goals about how much time they should spend one-on-one. This in turn, may require having a 
more explicit understanding with the centers and teachers about how the volunteers should be 
deployed. 


e To spend high-quality one-on-one time with children, volunteers should 
be intentional about the activities they engage in. 


Offering well-focused and language-rich CCT can be challenging even for a seasoned 
educator. Volunteers working in early childhood education centers may need additional training 
and support to make sure they are being intentional about helping their partner children build 
skills. For example, they might benefit from guidance on how to make the most of the one-on- 
one time. Supervisory staff could train the volunteers in how to engage their partner children with 


5Out of the 470 children who had consent from a parent to participate in the study, 93 children (20 percent) 
were not enrolled for the full academic year. 


Kontos and Wilcox-Herzog (1997). 
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activities that more consistently incorporate evidence-based strategies that build on and extend 
the child’s interest and learning. Such strategies could include scaffolding — volunteers support- 
ing and guiding children while they are learning something new and gradually decreasing the 
level of support as children’s abilities develop — as well as incorporating the child’s cultural and 
linguistic background into interactions. Supervisory staff could also demonstrate how to increase 
conversational exchanges with children during CCT and provide a toolkit of ideas for how to use 
one-on-one time. 


Another factor that may enhance volunteers’ ability to provide high-quality one-on-one 
time is their ability to communicate with children in their home language. In some cities, many 
children speak a language other than English at home. Deploying volunteers to centers based on 
the language they and the children speak is another strategy that may help promote stronger bonds 
between children and volunteers. 


Lessons for Future Studies 


e When used for evaluation purposes, data collected by teachers should be 
supplemented by additional training. 


In early childhood education studies, children’s outcomes are typically measured using 
teacher reports (based on an observational tool) and direct assessments. Teachers are generally 
trained in how to use an observational tool to make sure that the information collected is valid 
and reliable. This study is unique in that it used an existing source of data: a teacher-reported 
assessment called the Desired Results Developmental Profile — Preschool (DRDP-PS), instead of 
collecting child data specifically for the purposes of the study. Using this tool made it possible to 
collect data at a lower cost and put less of a burden on the study centers. Although the tool has 
been shown to have high internal and inter-rater reliability and validity when properly used, the 
pattern of DRDP-PS scores for children in the study centers suggests that teachers may not be 
using the tool as intended. For example, the pattern of differences in scores across classrooms 
suggests that teachers may be interpreting the scales and ratings in different ways. One would not 
expect the children in the CCT-only classrooms to make numerically larger (if not statistically 
significant) gains than children in the Jumpstart classrooms, because children in the Jumpstart 
classrooms receive both sessions and CCT (that is, CCT-only children receive fewer services). 


Future studies that measure child outcomes by using existing teacher observation tools 
should provide teachers with additional training (beyond what is provided by preschool centers), 
to make sure they are using the tools correctly and consistently. Additionally, if study resources 
allow it, direct assessments should be used as well so that information about children’s develop- 
ment can be assessed from multiple perspectives. 


e To reduce potential teacher bias in data on children’s outcomes, evalua- 
tors should consider using direct child assessment. 


When using direct assessments, a trained assessor sits with individual children and en- 
gages them in a series of activities and games designed to measure their development. The 
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advantage of direct assessments is that they are conducted by an independent observer who is not 
biased by their relationship with the child; the advantage of a teacher-reported observation tool is 
that teachers know the children better and are able to rate them based on observing them over 
multiple days (and not, like a direct assessment, at just one point in time). 


e To fully understand a program’s potential to improve child outcomes, 
studies should aim to observe volunteers directly when they are providing 
services. 


This study primarily relied on a teacher survey and data collected by Jumpstart to under- 
stand how the volunteers spent their time in the classrooms. However, these data sources provided 
a limited (and perhaps biased) range of perspectives on volunteers’ time and classroom practices. 
To better understand the effect of volunteers on children’s outcomes, future evaluations of 
Jumpstart or other volunteer programs should conduct formal classroom observations with valid 
and reliable tools to examine how the volunteers engage with children, the quality of literacy and 
language instructional practices they use, and how they support children and teachers in the class- 
room. Part of this data collection could include measuring how much time is spent providing one- 
on-one attention to partner children compared with how much time is spent providing general 
classroom support. Observational data could also be collected to better understand the ways in 
which the volunteers support social-emotional skills development in children. 


Looking Forward: Enhancements to the Jumpstart Model 


Jumpstart has made several enhancements to CCT and to the Jumpstart sessions, which were 
implemented by Jumpstart’s College Corps in fall 2019. The goal of these enhancements is to 
help volunteers provide more intentional and focused language and literacy and social-emotional 
supports to children. 


Volunteers are now trained in a modified version of CCT called Individual Classroom 
Service (ICS), to help them implement higher-quality, individualized classroom time with their 
partner children. ICS focuses on more intentional activities to promote language and literacy de- 
velopment. Volunteers receive training and support in the use of strategies for engaging children 
in conversations during one-on-one time, as well as activities for building vocabulary and com- 
prehension skills. During ICS, volunteers continue supporting the general classroom by assisting 
children during teacher-led activities (such as Circle Time), but there are guidelines that empha- 
size dedicated attention to partner children (that is, approximately 50 percent of volunteers’ 
nonsession service time should be spent providing individual attention). Making a clear distinc- 
tion between the two components of ICS (individualized learning activities and general classroom 
support) provides a new framework for organizing volunteers’ time. 


The Jumpstart session curriculum has been adapted to more strongly emphasize oral lan- 
guage and social-emotional skill-building. This increased focus on oral language creates coher- 
ence across the components of Jumpstart programming (sessions and ICS) and emphasizes a 
deepening of children’s knowledge-based competencies. The change also means that Jumpstart 
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has moved away from activities with an explicit focus on print knowledge and phonological 
awareness. While these activities, such as songs and chants, were sometimes easier for volunteers 
to implement, they were not necessarily leading to gains for children. With a deeper focus on oral 
language and social-emotional development, Jumpstart intends to better supplement the language 
and literacy instruction teachers are already providing, and drive impact in an area that research 
has identified as being critical for the population of children that Jumpstart serves.’ 


The findings from this study suggest that these modifications, which are intended to help 
the volunteers provide children with a substantially different instructional and supportive experi- 
ence, could further strengthen the Jumpstart model and lead to better outcomes for children in the 
communities served by the program. 


THoff (2013). 


50 


Appendix A 


Teacher Survey Instrument 


Jumpstart Teacher Survey 
Version: Jumpstart classroom teachers 
Spring 2018 


Thank you for completing the Teacher Survey as part of the Jumpstart Foster Grandparent Research Study. 


Instructions & Reminders: 
1. Please read and sign the consent form (on separate page) before getting started. 


2. Make sure there is an ID sticker in the upper right-hand corner of the first page of the survey. (If not, please let your su- 
pervisor know right away.) 


3. Remember: Your name will not be linked with your responses. Your consent form will be separated from the survey once 
MDRC receives it. Your individual responses will not be shared with Jumpstart. You may skip any questions you don’t want 
to answer but we would appreciate if you tried to answer as many questions as you can, as honestly as you can. 


Please use blue or black ink to complete the survey. 
Once you have completed the survey, please place the survey in the envelope provided and seal the envelope. Please do not 


place your consent form in the sealed envelope. 


Please hand deliver the completed survey in the sealed envelope and the separate consent form to your supervisor who is re- 
sponsible for making sure completed forms get to MDRC. By hand delivering the survey and the consent form, the liaison can 
remove your name from his/her follow-up list. 


Thank you for your help! 
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SECTION 1 -- BACKGROUND 


First, please tell us about your background. 


Q1. What is your age? 
(Please select one.) 
O, Under 25 
O, 25.29 
O; 30-39 
O; 40-49 
Os 50-59 


O; 60 or more 


Qz2. Are you of Hispanic, Latino or Spanish origin? 


(Please select one.) 
O,” Yes 
O; No 


Q3. What is your race? 
(Please select all that apply.) 


CL, Asian 
LI, Black or African American 


c Native American 


e White 


L] 
Ey Pacific Islander 
L 
L] 


f Something else (Please describe: ) 


Q4. How many years as an early childhood educator do you have? 
(Please round to the nearest year.) 


years 


Q5. How many years have you worked at this Early Childhood Center? 
(Please round to the nearest year.) 


years 


Q6. How much training to support children's language and literacy development have you had in the 2017-18 
school year? 


(Please round to the nearest hour.) 


hours 


Q7. What is the highest level of education you have attained? 
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Qs. 


Q9. 


(Please select one.) 

O, Some High School 

©, High School or GED 

O; Child Development Associate 

O; Some College 

O; Associate's Degree 

O; Bachelor's Degree 

O, Graduate Degree 

O; Some other degree (Please describe: 


ay 


Last week, about how much time did you spend preparing language and literacy activities for your students? 


hours in the last week and minutes in the last week 


How much do you agree or disagree with the following statements: 


(Please select one for each statement.) Neither Decline 
Strongly Agree nor Strongly to An- 
Disagree Disagree Disagree Agree Agree swer 
The amount a child can learn is primarily related O 
to family background. 7 . O: O: Os O. Os ‘ 
A teacher is very limited in what he/she can 
achieve because a child’s home environment is a oO, 2 3 4 5 
large influence on his/her achievement. 
If parents would do more for their children, I O 
d: 2 3 4 5 


could do more. 


If a child in my class becomes disruptive and 
noisy, I feel assured that I know some techniques O; 
to redirect him/her quickly. 


6 


Ifa child couldn't do a class activity, I would be 
able to accurately assess whether the activity was ©, 


O 
e 
O 
O 


6 


at the correct level of difficulty. 
If I really try hard, I can get through to even the O C . 


most difficult or unmotivated children. . 


When it comes right down to it, a teacher really 

can’t do much because most of a child’s motiva- O 
. . 1 

tion and performance depends on his/her home 

environment. 


a 


O,; 


SECTION 2 -JUMPSTART SESSION TIME 


The following section asks you to consider language and literacy instructional practices that take place in your classroom during 
the Jumpstart sessions and identify how often, if ever, the Foster Grandparent volunteers assigned to your classroom utilize each 
practice. 


When responding to the next question, please consider how often, on average, the volunteers in your classroom engaged in each 
practice throughout the Jumpstart sessions that took place last week. 


Q10. Last week, how often did the Foster Grandparent volunteers assigned to your classroom do the following dur- 
ing the Jumpstart sessions? 


(Please select one for each statement.) Decline 
Some- Almost to An- 
Never Seldom times Often Always swer 
Created a variety of opportunities for children to 
Created a variety of opp : Oo a a oa @ 
interact in one-on-one conversations. 
Created a variety of opportunities for children to 
ty of opp ao a a o 


interact in small group conversations. 


Initiated conversations that maximized talk and 

built on oral language skills. 

(For example: Had a discussion about how the class ©; ©; ©, ©, O. O. 
could create their own post office, including brain- 

storming and listing the supplies they would need, etc.) 


Asked open-ended questions to children for which 

an answer was not known (and that require more 

than a one- or two-word response) that would en- O 
courage them to talk about their thinking. O,: O; O; O;: O. i 
(For example: “Tell me about our day at the zoo” in- 

stead of “What color was the flamingo?”) 


Talked to the children in my classroom 


using some words slightly beyond their 

understanding to broaden their OO; Sy O; OO; ©. Os 
knowledge of words and sentence 

structure. 

G - iate definiti f ds to chil- 

alias appropriate definitions of words to chi O, O, O, O, CO): eo 
Used songs, chants, nursery rhymes, and rhyming O O O O O ©, 
poems. * . : 3 

Played games that got children to listen for begin- 

ning and ending sounds. O 
(For example: “I spy something that begins with a /b/ O, O, O; O: O; : 
sound.”) 

During reading, used expressions, tone of voice, 

gestures and/or references to illustrations to de- O O O O O eo 
fine words and support children's understanding : . : ? 

of the story. 

Used song and word games to isolate sounds in 

words. 

(For example: “What is the sound that begins these ). e ©, ©; ©). O. 


words: chin, chicken, and cheek? Ch is the sound that be- 
gins these words.”) 


56 


(Please select one for each statement.) 


Decline 


Some- Almost to An- 
Never Seldom times Often Always swer 
Helped children connect letters and sounds. O, oF O; Oi; er eo 


Now we would like to ask you about each Jumpstart session component. 


Q11. 


Q12. 


Please tell us how much you agree or disagree that the following took place during the Welcome component 


of the Jumpstart sessions last week: 


(Please indicate the extent to which you agree or disagree with each statement.) 


Neither 
Agree Decline 
Strongly nor Disa- Strongly to An- 
Disagree Disagree gree Agree Agree swer 
Children transitioned to the Jumpstart session O, O, os O, O. eo 
from their previous activity smoothly. 
Children built alphabet knowledge through ex- O. O, O, O. O. ©, 


ploration of name cards. 


Please tell us how much you agree or disagree that the following took place during the Reading component of 


the Jumpstart sessions last week: 


(Please indicate the extent to which you agree or disagree with each statement.) 


Neither 
Agree Decline 
Strongly nor Disa- Strongly to An- 
Disagree Disagree gree Agree Agree swer 
Most children were engaged in a shared reading ©. O, ©; ©; ©: ee 
experience. 
Foster Grandparents introduced children to a va- O. O, ©. ©. CS. e@ 


riety of rich vocabulary words. 


Foster Grandparents asked children questions 
about a story to support the development of their O; 
comprehension skills. 
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O; 


O; 


O,” 


O, 


Q13. Please tell us how much you agree or disagree that the following took place during the Circle Time component 


of the Jumpstart sessions last week: 


(Please indicate the extent to which you agree or disagree with each statement.) 


Strongly 
Disagree 


Disagree 


Neither 


Agree 


nor Disa- 


gree 


Agree 


Strongly 
Agree 


Decline 
to An- 
swer 


Children had the opportunity to participate, 
make suggestions, and practice taking turns. 


O, 


O; 


O; 


O: 


O; 


O; 


Activities (songs, poems, games) were offered at 
an appropriate level for children. 


Children were able to participate at their own 
level. 


O” 


O” 


O; 


O; 


O; 


O; 


O,” 


O,” 


O. 


O. 


O,; 


O; 


Q14. Please tell us how much you agree or disagree that the following took place during the Center Time compo- 


nent of the Jumpstart sessions last week: 


(Please indicate the extent to which you agree or disagree with each statement.) 


Strongly 


Disagree 


Disagree 


Neither 
Agree 


nor Disa- 


gree 


Agree 


Strongly 
Agree 


Decline 
to An- 
swer 


Most children were engaged in Center Time activ- 


ities. 


O” 


O; 


O; 


O,” 


O. 


O; 


The centers that were set up involved activities 
that support children’s language and literacy 
skill development. 


O, 


O, 


O; 


O;: 


O, 


Os 


Volunteers used rich, concrete language that 
helped children to build their vocabulary. 


Volunteers played as partners with children. 


Centers were set up with materials and activities 
that were appropriate for young children. 
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Q15. Please tell us how much you agree or disagree that the following took place during the Let’s Find Out About It 
component of the Jumpstart sessions last week: 


(Please indicate the extent to which you agree or disagree with each statement.) 


Neither 
Agree Decline 
Strongly nor Disa- Strongly to An- 
Disagree Disagree gree Agree Agree swer 
Most children were engaged in the small group '@ O, oC. O, O. ©, 
activities. 
Children learned about objects and their use O; O, O, ©), C). ©, 
through the small group activities. 
The small group activities supported children’s O, O, OC), O, O. ©, 


language and literacy skill development. 


Q16. Please tell us how much you agree or disagree that the following took place during the Sharing and Goodbye 
component of the Jumpstart sessions last week: 


(Please indicate the extent to which you agree or disagree with each statement.) 


Neither 
Agree Decline 
Strongly nor Disa- Strongly to An- 
Disagree Disagree gree Agree Agree swer 
Some children shared their favorite session activ- O, O, Cy O, CE eo 
ities in the large group conversation. 
Foster Grandparents used objects or examples of 
children’s work from Center Time to support the s) CG) es OC), (S: Os 
large group conversation. 
Foster Grandparents used rich vocabulary to sup- ©; O, O, O. O. eo 


port the large group conversation. 
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SECTION 3 - NON-JUMPSTART SESSION TIME 


The following section asks you to consider language and literacy instructional practices that take place in your classroom outside 
of the Jumpstart sessions. 


First, we would like to ask you about the supports the Foster Grandparent volunteers provide outside of the Jumpstart sessions. 


Q17. Last week, what kinds of one-on-one activities did the Foster Grandparent volunteers assigned to your class- 
room engage in with their partner children outside of the Jumpstart sessions? 


(Please select all that apply.) 


LI, — Read a book to the child 

LI, Practiced recognizing letters 

LJ, — Practiced recognizing numbers 

CIs Drewor completed other arts and crafts projects 

Ed Sang a song or made music 

L1; — Used building blocks 

LJ, Played dress-up 

L, — Played with dolls or other toys 

CL Supported children during transitions 

L} Engaged in conversations or supported children during snack or meal time 

CI, Something else (Please describe:__ =} 
Q18. Last week, during these activities, what types of language or literacy strategies did the Foster Grandparent 

volunteers assigned to your classroom use with their partner children? 


(Please select all that apply.) 


aE Directly taught new words to children (For example: “A cave is a hole in the mountain.”) 


LJ, — Connected a new word to a word the children already know (For example: “Enormous - it means re- 
ally big. Say ‘enormous’ with me.”) 


C1, — Had children repeat new words by asking them to respond to prompt (For example: “What does ___ 
mean?”) 


LJ, Asked children comprehension questions about a story 

Baile Helped children say sounds in unfamiliar words 

C1, Embedded the use of new words into children’s daily experiences 

LJ; — Noted specific features of print and letters (e.g., “That is the letter D like Deondre’s name”) 
L], — Asked children where to start reading 

LI; Counted words in a text with children 

L] Pointed out print within pictures 

LI, — Something else (Please describe: ) 
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Q19. Please use this space to describe how volunteers interact with their partner children and provide supports in 
your classroom. 


Now we would like to ask you about the language and literacy instructional practices YOU utilize. 


Q20. Last week, how many days did you spend on language and literacy instruction and/or activities with your 
children outside of the Jumpstart sessions? 


(Please round to the nearest day.) 
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When responding to the next question, please consider how often, on average, you engaged in each practice outside of the 
Jumpstart sessions during the last week. 


Q21. Last week, how often did you do the following outside of the Jumpstart sessions? 


(Please select one for each statement.) Decline 
Some- Almost to An- 
Never Seldom times Often Always swer 
Created i f tunities for children t 
reated a variety of opportunities for children to O; ey O, ©; O. eo 


interact in one-on-one conversations. 


Created a variety of opportunities for children to O O O O O eo 
. . . 1 2 3 4 5 
interact in small group conversations. 


Initiated conversations that maximized talk and 

built on oral language skills. 

(For example: Had a discussion about how the class ©: C Oo,” ay O; O; 
could create their own post office, including brain- 

storming and listing the supplies they would need, etc.) 


Asked open-ended questions to children for 

which an answer was not known (and that re- 

quire more than a one- or two-word response) 

that would encourage them to talk about their ©: ar er © , O; O; 
thinking. 

(For example: “Tell me about our day at the zoo” in- 

stead of “What color was the flamingo?”) 


Talked to the children in my classroom using 

some words slightly beyond their understanding c) O O O O ©, 
to broaden their knowledge of words and sen- * 2 : i ; 

tence structure. 


Gave age-appropriate definitions of words to O O O O O eo 
children. : : ° : : 

Used songs, chants, nursery rhymes, and rhyming O. O, O7 O, ©): eo 
poems. 


Played games that got children to listen for begin- 
ning and ending sounds. O 
(For example: “I spy something that begins with a /b/ O, O, O; O: O, i 


sound.”) 


During reading, used expressions, tone of voice, 

gestures and/or references to illustrations to de- O O © © O eo 
. . 1 . 1 2 3 4 5 

fine words and support children's understanding 

of the story. 


Used song and word games to isolate sounds in 


words. 

(For example: “What is the sound that begins these ©, oF ©, O, ©. O. 
words: chin, chicken, and cheek? Ch is the sound that 

begins these words.”) 


Helped children connect letters and sounds. oO,” 2; ss O, S: Os 


Now we would like to ask you about the environment in your classroom outside of the Jumpstart sessions. 
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Q22. 


Q23. 


Last week, how often did children in your classroom do the following outside of the Jumpstart sessions? 


(Please select one for each statement.) 


Moved smoothly throughout this period of day 
with few conflicts. 


Never 


O” 


Seldom 


O; 


Some- 
times 


O; 


Often 


O,” 


Almost 
Always 


Decline 
to An- 
swer 


Needed to be reminded of rules and routines. 


O” 


O; 


O; 


O,” 


Did off-task things. 


O, 


O; 


O; 


O;: 


Created behavior problems in your class. 


O, 


O, 


O; 


O: 


Followed established classroom rules and proce- 
dures. 


Please use this space for additional comments about language and literacy practices in your classroom: 


O, 


O, 


O; 


O: 


Thank you for completing the survey! 
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Jumpstart Teacher Survey 
Version: CCT-only classroom teachers 
Spring 2018 


Thank you for completing the Teacher Survey as part of the Jumpstart Foster Grandparent Research Study. 


Instructions & Reminders: 
1. Please read and sign the consent form (on separate page) before getting started. 


2. Make sure there is an ID sticker in the upper right-hand corner of the first page of the survey. (If not, please let your su- 
pervisor know right away.) 


3. Remember: Your name will not be linked with your responses. Your consent form will be separated from the survey once 
MDRC receives it. Your individual responses will not be shared with Jumpstart. You may skip any questions you don’t want 
to answer but we would appreciate if you tried to answer as many questions as you can, as honestly as you can. 


Please use blue or black ink to complete the survey. 
Once you have completed the survey, please place the survey in the envelope provided and seal the envelope. Please do not 


place your consent form in the sealed envelope. 


Please hand deliver the completed survey in the sealed envelope and the separate consent form to your supervisor who is re- 
sponsible for making sure completed forms get to MDRC. By hand delivering the survey and the consent form, the liaison can 
remove your name from his/her follow-up list. 


Thank you for your help! 
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SECTION 1 -- BACKGROUND 


First, please tell us about your background. 


Q1. What is your age? 
(Please select one.) 
O, Under 25 
O, 25.29 
O; 30-39 
O; 40-49 
Os 50-59 


O; 60 or more 


Qz2. Are you of Hispanic, Latino or Spanish origin? 


(Please select one.) 
O,” Yes 
O; No 


Q3. What is your race? 
(Please select all that apply.) 


CL, Asian 
LI, Black or African American 


c Native American 


e White 


L] 
Ey Pacific Islander 
L 
L] 


f Something else (Please describe: ) 


Q4. How many years as an early childhood educator do you have? 
(Please round to the nearest year.) 


years 


Q5. How many years have you worked at this Early Childhood Center? 
(Please round to the nearest year.) 


years 


Q6. How much training to support children's language and literacy development have you had in the 2017-18 
school year? 


(Please round to the nearest hour.) 


hours 
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Q7. 


Qs. 


Q9. 


Q10. 


What is the highest level of education you have attained? 


(Please select one.) 

O, Some High School 

©, — High School or GED 

O; Child Development Associate 

O, Some College 

‘S) Associate's Degree 

O; Bachelor's Degree 

O, Graduate Degree 

'S) Some other degree (Please describe: 


J 


Last week, about how much time did you spend preparing language and literacy activities for your students? 


hours in the last week and 


minutes in the last week 


Last week, how many days did you spend on language and literacy instruction and/or activ- 


ities with your children? 


(Please round to the nearest day.) 


How much do you agree or disagree with the following statements: 


(Please select one for each statement.) 


Strongly 
Disagree 


Disagree 


Neither 
Agree nor 
Disagree 


Agree 


Strongly 
Agree 


Decline 
to An- 
swer 


The amount a child can learn is primarily related 
to family background. 


A teacher is very limited in what he/she can 
achieve because a child’s home environment is a 
large influence on his/her achievement. 


If parents would do more for their children, I 
could do more. 


If a child in my class becomes disruptive and 
noisy, I feel assured that I know some techniques 
to redirect him/her quickly. 


Ifa child couldn't do a class activity, I would be 
able to accurately assess whether the activity was 
at the correct level of difficulty. 


If I really try hard, I can get through to even the 
most difficult or unmotivated children. 


When it comes right down to it, a teacher really 
can’t do much because most of a child’s motiva- 
tion and performance depends on his/her home 
environment. 
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SECTION 2 — LANGUAGE AND LITERACY TIME 


The following section asks you to consider language and literacy instructional practices that take place in 
your Classroom and identify how often, if ever, YOU utilize each practice. 


When responding to the next question, please consider how often, on average, you engaged in each practice. 


Q11. Last week, how often did you do the following during language and literacy instructional time? 


(Please select one for each statement.) Decline 
Some- Almost to An- 
Never Seldom times Often Always swer 
Created i f tunities for children t 
reated a variety of opportunities for children to ©; O, O, O, O, eo 


interact in one-on-one conversations. 


Created a variety of opportunities for children to O O O O O eo 
. . . 1 2 3 4 5 
interact in small group conversations. 


Initiated conversations that maximized talk and 

built on oral language skills. 

(For example: Had a discussion about how the class oF Ci OC OC, ©: Os 
could create their own post office, including brain- 

storming and listing the supplies they would need, etc.) 


Asked open-ended questions to children for 

which an answer was not known (and that re- 

quire more than a one- or two-word response) 

that would encourage them to talk about their ©; ©; ©, ©: Cc): O; 
thinking. 

(For example: “Tell me about our day at the zoo” in- 

stead of “What color was the flamingo?”) 


Talked to the children in my classroom using 

some words slightly beyond their understanding O O 'S O O ©, 
to broaden their knowledge of words and sen- = : ? ° ° 

tence structure. 


Gave age-appropriate definitions of words to O O ©) © O O, 
children. * 7 : ° 

Used songs, chants, nursery rhymes, and rhyming O. O, O~ O, ©; O, 
poems. 


Played games that got children to listen for begin- 

ning and ending sounds. O 
(For example: “I spy something that begins with a /b/ O, O, O; O: O; : 
sound.”) 


During reading, used expressions, tone of voice, 

gestures and/or references to illustrations to de- O O O O O eo 
. . 1 . 1 2 3 4 5 

fine words and support children's understanding 

of the story. 


Used song and word games to isolate sounds in 
words. 
(For example: “What is the sound that begins these oO” oO,” O," O,~ ©): O, 


words: chin, chicken, and cheek? Ch is the sound that 
begins these words.”) 
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(Please select one for each statement.) Decline 


Some- Almost to An- 
Never Seldom times Often Always swer 
Helped children connect letters and sounds. 0; > Co; O, O; O: 


Now we would like to ask you about the general environment in your classroom during language and literacy instructional time. 


Q12. Ingeneral, how often did children in your classroom do the following last week during language and literacy 
instructional time? 


(Please select one for each statement.) Decline 
Some- Almost to An- 

Never Seldom times Often Always swer 

Moved smoothly throughout this period of day O, O, O, O; O. O, 


with few conflicts. 


Needed to be reminded of rules and routines. Oi} es O, oO,” es 


Did off-task things. ©); O, O; C, ©: 


Created behavior problems in your class. O; OC; Oj ©; O; 2 
— established classroom rules and rou- oO,” CO), ©. ©; O, 6 


Now, we would like to ask you about the supports the Foster Grandparent volunteers assigned to your classroom provided dur- 
ing language and literacy instructional time. 


Q13. Last week, what kinds of one-on-one activities did the Foster Grandparent volunteers assigned to your class- 
room engage in with their partner children? 


(Please select all that apply.) 


LJ, Read a book to the child 

LI, — Practiced recognizing letters 

L), — Practiced recognizing numbers 

LI; — Drewor completed other arts and crafts projects 

(ale Sang a song or made music 

L1; — Used building blocks 

LI], Played dress-up 

CI, Played with dolls or other toys 

LC); Supported children during transitions 

L Engaged in conversations or supported children during snack or meal time 
LI, Something else (Please describe:__ iii‘ ) 
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Q14. 


Q15. 


Last week, during these activities, what types of language or literacy strategies did the Foster Grandparent 
volunteers assigned to your classroom use with their partner children? 


(Please select all that apply.) 


Directly taught new words to children (For example: “A cave is a hole in the mountain”) 


Connected a new word to a word the children already know (For example: “Enormous - it means re- 
ally big. Say ‘enormous’ with me.”) 


Had children repeat new words by asking them to respond to prompt (For example: “What does ___ 
mean?”) 


Asked children comprehension questions about a story 

Helped children say sounds in unfamiliar words 

Embedded the use of new words into children’s daily experiences 

Noted specific features of print and letters (e.g., “That is the letter D like Deondre’s name”) 
Asked children where to start reading 

Counted words in a text with children 

Pointed out print within pictures 

Something else (Please describe:____ _) 


Please use this space to describe how volunteers interact with their partner children and provide supports in 
your classroom. 
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Q16. Last week, how often did children in your classroom do the following specifically when volunteers were pre- 
sent in your classroom? 


(Please select one for each statement.) Decline 
Some- Almost to An- 
Never Seldom times Often Always swer 

Moved thly th hout thi iod of d 
oved smoothly throughout this period of day O, O, O, O, O. ©, 


with few conflicts. 


Needed to be reminded of rules and routines. ©), O, OO; O, 


O 
Did off-task things. Oo & O&O OD ©: 
O 
O 


Created behavior problems in your class. O, OQ; O; ), 5 
nei established classroom rules and rou- O, ©, O, O; : 6 


Q17. Please use this space for additional comments about language and literacy practices in your classroom: 


Thank you for completing the survey! 
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Jumpstart Teacher Survey 
Version: Business as usual classroom teachers 
Spring 2018 


Thank you for completing the Teacher Survey as part of the Jumpstart Foster Grandparent Research Study. 


Instructions & Reminders: 
1. Please read and sign the consent form (on separate page) before getting started. 


2. Make sure there is an ID sticker in the upper right-hand corner of the first page of the survey. (If not, please let your su- 
pervisor know right away.) 


3. Remember: Your name will not be linked with your responses. Your consent form will be separated from the survey once 
MDRC receives it. Your individual responses will not be shared with Jumpstart. You may skip any questions you don’t want 
to answer but we would appreciate if you tried to answer as many questions as you can, as honestly as you can. 


Please use blue or black ink to complete the survey. 
Once you have completed the survey, please place the survey in the envelope provided and seal the envelope. Please do not 


place your consent form in the sealed envelope. 


Please hand deliver the completed survey in the sealed envelope and the separate consent form to your supervisor who is re- 
sponsible for making sure completed forms get to MDRC. By hand delivering the survey and the consent form, the liaison can 
remove your name from his/her follow-up list. 


Thank you for your help! 
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SECTION 1 -- BACKGROUND 


First, please tell us about your background. 


Q1. What is your age? 
(Please select one.) 
O, Under 25 
O, 25.29 
O; 30-39 
O; 40-49 
Os 50-59 


O; 60 or more 


Qz2. Are you of Hispanic, Latino or Spanish origin? 


(Please select one.) 
O,” Yes 
O; No 


Q3. What is your race? 
(Please select all that apply.) 
CL, Asian 
El Black or African American 
im Native American 
LI, Pacific Islander 


LI, White 


CI, Something else (Please describe: ) 


Q4. How many years as an early childhood educator do you have? 
(Please round to the nearest year.) 


years 


Q5. How many years have you worked at this Early Childhood Center? 
(Please round to the nearest year.) 


years 


Q6. How much training to support children's language and literacy development have you had in the 2017-18 
school year? 


(Please round to the nearest hour.) 


hours 


da 


Q7. 


Qs. 


Q9. 


Q10. 


What is the highest level of education you have attained? 


(Please select one.) 

O, Some High School 

©, High School or GED 

O; Child Development Associate 

O, Some College 

‘S) Associate's Degree 

O; Bachelor's Degree 

O, Graduate Degree 

'S) Some other degree (Please describe: 


J 


Last week, about how much time did you spend preparing language and literacy activities for your students? 


hours in the last week and 


minutes in the last week 


Last week, how many days did you spend on language and literacy instruction and/or activ- 


ities with your children? 


(Please round to the nearest day.) 


How much do you agree or disagree with the following statements: 


(Please select one for each statement.) 


Strongly 
Disagree 


Disagree 


Neither 
Agree nor 
Disagree 


Agree 


Strongly 
Agree 


Decline 
to An- 
swer 


The amount a child can learn is primarily related 
to family background. 


A teacher is very limited in what he/she can 
achieve because a child’s home environment is a 
large influence on his/her achievement. 


If parents would do more for their children, I 
could do more. 


If a child in my class becomes disruptive and 
noisy, I feel assured that I know some techniques 
to redirect him/her quickly. 


Ifa child couldn't do a class activity, I would be 
able to accurately assess whether the activity was 
at the correct level of difficulty. 


If I really try hard, I can get through to even the 
most difficult or unmotivated children. 


When it comes right down to it, a teacher really 
can’t do much because most of a child’s motiva- 
tion and performance depends on his/her home 
environment. 


fe 


SECTION 2 — LANGUAGE AND LITERACY TIME 


The following section asks you to consider language and literacy instructional practices that take place in your classroom and 
identify how often, if ever, YOU utilize each practice. 


When responding to the next question, please consider how often, on average, you engaged in each practice. 


Q11. Last week, how often did you do the following during language and literacy instructional time? 


(Please select one for each statement.) Decline 
Some- Almost to An- 

Never Seldom times Often Always swer 

Created a variety of opportunities for children to ©) O, O, CC) O. ©, 


interact in one-on-one conversations. 


Created a variety of opportunities for children to O O O O O O, 
. . . 1 2 3 4 5 
interact in small group conversations. 


Initiated conversations that maximized talk and 

built on oral language skills. 

(For example: Had a discussion about how the class ©; oO,” Oo,” O, ©: e 
could create their own post office, including brain- 

storming and listing the supplies they would need, etc.) 


Asked open-ended questions to children for 

which an answer was not known (and that re- 

quire more than a one- or two-word response) 

that would encourage them to talk about their ©: oO,” Oo,” O hi O; O; 
thinking. 

(For example: “Tell me about our day at the zoo” in- 

stead of “What color was the flamingo?”) 


Talked to the children in my classroom using 

some words slightly beyond their understanding O Se O O O eo 
to broaden their knowledge of words and sen- . . : A 

tence structure. 


Gave age-appropriate definitions of words to O O O O O eo 
children. : . : : : 

Used songs, chants, nursery rhymes, and rhyming CO, ©, O7 O, O. eo 
poems. 


Played games that got children to listen for begin- 

ning and ending sounds. O 
(For example: “I spy something that begins with a /b/ O, O, O; O;: O. 
sound.”) 


During reading, used expressions, tone of voice, 

gestures and/or references to illustrations to de- O O O O O eo 
- a ' . 1 2 3 4 5 

fine words and support children's understanding 

of the story. 


Used song and word games to isolate sounds in 


words. 

(For example: “What is the sound that begins these ©; Oo,” Oo,” O, CC: @ 
words: chin, chicken, and cheek? Ch is the sound that 

begins these words.”) 


Helped children connect letters and sounds. OO: ); @) 2; es Os 
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Now we would like to ask you about the general environment in your classroom during language and literacy instructional time. 


Q12. 


Q13. 


In general, how often did children in your classroom do the following last week during language and literacy 


instructional time? 


(Please select one for each statement.) 


Never 


Seldom 


Some- 
times 


Often 


Almost 
Always 


Decline 
to An- 
swer 


Moved smoothly throughout this period of day 
with few conflicts. 


O, 


O, 


O; 


O: 


O; 


Needed to be reminded of rules and routines. 


O, 


O; 


O; 


O: 


O; 


Did off-task things. 


O, 


O; 


O; 


O: 


O; 


Created behavior problems in your class. 


O, 


O, 


O; 


O: 


O; 


Followed established classroom rules and rou- 
tines. 


Please use this space for additional comments about language and literacy practices in your classroom: 


O, 


O, 


O; 


O: 


O; 


Thank you for completing the survey! 
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Appendix B 


Recruitment and Sample 


Figure B.1 


Sample for the Analysis of the Jumpstart Foster Grandparent Evaluation 


647 children enrolled at centers. 


555 returned parental consent 
forms. 


470 parents gave consent to have 
children participate in study. 


394 children had spring DRDP-PS 
assessments (the semester of the 
primary outcomes). 


336 children had both fall and 
spring DRDP-PS assessments 
and were assessed in the fall 
before December 1, 2017. 


105 children enrolled in one of the 
five centers that allowed children 
to be randomly assigned across 
three types of classrooms. 
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Appendix C 


The Desired Results Developmental Profile — Preschool 
(DRDP-PS) Measures 


The Desired Results Development Profile — Preschool (DRDP-PS) is a teacher-reported, formative assess- 
ment. The DRDP is made up of eight domains: Approaches to Learning—Self-Regulation; Social and Emo- 
tional Development (which measures children’s interpersonal skills); Language and Literacy Development 
(which rates children’s development in their main language); English-Language Development (for dual 
language learners only); Cognition; Physical Development-Health; History-Social Science; Visual and Per- 
forming Arts. The nine levels are: Responding (earlier and later), Exploring (earlier, middle, later), Building 
(earlier, middle, later), and Integrating (earlier). ! 


Impact Outcome Measures Constructed from DRDP domains 


For the impact study, the research team used two outcome measures based on children’s scores in four 
specific domains measured by the DRDP: 


The English Language and Literacy Development Outcome Measure 


The English Language and Literacy Development outcome that is used in this evaluation is based 
on children’s scores on two DRDP domains. For children who are not dual language learners, their English 
Language and Literacy Development score is based on their ratings on items in the DRDP’s “Language 
and Literacy Development” domain. This domain assesses the progress of children in developing founda- 
tional language and literacy skills. These skills can be demonstrated in any language and in any mode of 
communication, and dual language learners may demonstrate knowledge and skills in their home language. 
Therefore, for children who are dual language learners, their score for the DRDP’s English-Language De- 
velopment domain is used instead, because this assesses their progress toward learning to communicate in 
English. 


The Language and Literacy domain includes 10 measures. For each one, teachers rate the develop- 
mental level of children on a nine-point scale: (1) Responding Earlier, (2) Responding Later, (3) Exploring 
Earlier, (4) Exploring Middle, (5) Exploring Later, (6) Building Earlier, (7) Building Middle, (8) Building 
Later, and (9) Integrating Earlier. The measures of Language and Literacy Development domain are as 
follows: 


Measures of Language and Literacy Development 
e Understanding of Language 

e Responsiveness to Language 

e Communication and Use of Language 

e Reciprocal Communication and Conversation 


e Interest in Literacy 


‘A detailed description of DRDP-PS domains, measures, rating scales, and administration guidelines can be found here: 
https://www.desiredresults.us/sites/default/files/docs/forms/DRDP2015PSC_090116.pdf. 
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Comprehension of Age-Appropriate Text 
Concepts about Print 

Phonological Awareness 

Letter and Word Knowledge 


Emergent Writing 


The English-Language Development domain includes four measures. For each one, teachers rate 
the developmental level of children on a six-point scale: (1) Discovering Language, (2) Discovering Eng- 
lish, (3) Exploring English, (4) Developing English, (5) Building English, and (6) Integrating English. The 
measures of English Language Development are as follows: 


Measures of English-Language Development 


Comprehension of English 
Self-Expression in English 
Understanding and Response to English Literacy Activities 


Symbol, Letter, and Print Knowledge in English 


Social-Emotional Development Outcome Measure 


The Social-Emotional Development outcome that is used in this evaluation is a combined construct 
of scores from two domains in the DRDP: Interpersonal Skills (which is referred to as “Social and Emo- 
tional Development” in the DRDP but was renamed by the study team for this report) and Self-Regulation 
(which is referred to as Approaches to Learning—Self-Regulation in the DRDP). 


The Interpersonal Skills domain includes five measures. For each one, teachers rate the develop- 
mental level of children on a nine-point scale: (1) Responding Earlier, (2) Responding Later, (3) Exploring 
Earlier, (4) Exploring Middle, (5) Exploring Later, (6) Building Earlier, (7) Building Middle, (8) Building 
Later, and (9) Integrating Earlier. The measures of the Interpersonal Skills domain are as follows: 


Measures of Interpersonal Skills 


Identity of Self in Relation to Others 

Social and Emotional Understanding 

Relationships and Social Interactions with Familiar Adults 
Relationships and Social Interactions with Peers 


Symbolic and Sociodramatic Play 


84 


The Self-Regulation domain includes seven measures. Three of them are conditional measures that 
are only rated when a child has an individualized education plan or is a dual language learner. Theses con- 
ditional measures were not included in the composite social and emotional learning measure. Teachers rate 
the developmental level of children on a nine-point scale: (1) Responding Earlier, (2) Responding Later, 
(3) Exploring Earlier, (4) Exploring Middle, (5) Exploring Later, (6) Building Earlier, (7) Building Middle, 
(8) Building Later, and (9) Integrating Earlier. The measures of the Self-Regulation domain are as follows: 


Measures of Self-Regulation 

e Cunosity and Initiative in Learning 

e Self-Control of Feelings and Behavior 
e Engagement and Persistence 


e Shared Use of Space and Materials 


Methodology for creating outcome measures 


For each outcome, a composite score was created using a partial credit Rasch model for polytomous items. 
Using a partial credit Rasch model, instead of taking the average score across items, takes into account the 
variation in difficulty across items, giving a child more credit for being further developed on a difficult 
item. 


>The data were scaled using the Item Response Theory — Partial Credit Model package in Stata. 
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Appendix D 


Service Contrast for Five Impact Study Centers 


Table D.1 


Comparing Teacher and Foster Grandparent Use of Language and Literacy 
Instructional Practices in Jumpstart Classrooms, Impact Sample 


Foster 
Practices (5-point scale)* Grandparent Teacher 
Created opportunities for children to interact in 4.2 47 
one-on-one conversations 
Created opportunities for children to interact in 4.5 45 
small group conversations 
Initiated conversations that maximized talk and built 4.0 4.7 
on oral language skills 
Asked open-ended questions that would encourage 4.3 4.8 
children to talk about their thinking 
Talked to children using words slightly beyond their 4.7 4.3 
understanding to broaden knowledge of words 
Gave age-appropriate definitions of words 4.3 4.8 
Used songs, chants, nursery rhymes, and poems 4.8 4.8 
Played games that got children to listen for 4.3 4.3 
beginning and ending sounds 
Used expressions, tone of voice, gestures, or references 4.5 4.8 
to illustrations to support understanding during reading 
Used song and word games to isolate sounds in words 45 4.8 
Helped children connect letters and sounds 4.5 4.7 


SOURCE: MDRC calculations based on teacher surveys distributed and collected in spring 2018. 


NOTES: Findings in this table are based on data for 6 Jumpstart group teachers. Sample size for 
individual items varies due to nonresponse or skipped questions. 

aThe values presented in this table are calculated from the mean of a 5-point scale: 1 (never), 
2 (seldom), 3 (sometimes), 4 (often), and 5 (almost always). 
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Table D.2 


Percentage of Classrooms Where Foster Grandparents Employed 
Specific One-on-One Activities during CCT, Impact Sample 


Activities Percentage 
Read a book 72.7 
Practiced recognizing letters 81.8 
Practiced recognizing numbers 63.6 
Completed arts and crafts projects 63.6 
Sang songs or made music 63.6 
Used building blocks 63.6 
Played dress-up 63.6 
Played with dolls or toys 54.6 
Supported children during transitions 81.8 
Engaged children during snack or meals 81.8 
Sample size 11 


SOURCE: MDRC calculations based on teacher surveys distributed and collected in spring 2018. 


NOTES: Findings in this table are based on data for 5 Jumpstart group teachers and 6 CCT-only group 
teachers. Sample size for individual items varies due to nonresponse or skipped questions. 
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Appendix E 


Impacts for Combined Analytical Sample 
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Appendix F 


Teacher Characteristics Tables 


Table F.1 


Baseline Teacher Characteristics, Combined Study Sample 


Jumpstart CCT-Only Business as 
Characteristic (%) Group Group Usual Group 
Age 
Under 25 0.0 0.0 0.0 
25-29 0.0 0.0 0.0 
30-39 25.0 14.3 16.7 
40-49 16.7 57.1 ** 0.0 
50-59 58.3 28.6 66.7 
60 or more 0.0 0.0 16.7 
Hispanic, Latino, or Spanish 58.3 57.1 50.0 
Race 
Asian 0.0 0.0 25.0 
Black or African American 50.0 25.0 50.0 
Native American 0.0 0.0 0.0 
Pacific Islander 0.0 0.0 0.0 
White 40.0 12.5 25.0 
Other 20.0 62.5 ** 0.0 
Highest Education Level 
Some high school 0.0 0.0 0.0 
High school or GED 0.0 0.0 0.0 
Child Development Associate 8.3 0.0 0.0 
Some college 8.3 0.0 0.0 
Associate's degree 8.3 0.0 16.7 
Bachelor's degree 58.3 100.0 83.3 
Graduate degree 8.3 0.0 0.0 
Other 8.3 0.0 0.0 
Years as an early childhood educator 18.8 14.2 * 24.0 
Years at current early childhood center 9.3 7.6 12.5 
Sample size 12 9 6 


SOURCE: MDRC calculations based on teacher surveys distributed and collected in spring 2018. 


NOTES: Findings in this table are based on data for 12 Jumpstart group teachers, 9 CCT-only group 
teachers, and 6 comparison group teachers. Sample size for individual items varies due to nonresponse or 
skipped questions. Responses may add up to more that 100 percent as teachers "checked all that applied" 
for certain questions. 

A t-test was used to assess the statistical significance of each estimated difference with significance 
levels as follows: ***=1 percent, **=5 percent, and *=10 percent. Both Jumpstart and CCT-only groups were 
compared to the business as usual baseline. 
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Table F.2 


Baseline Teacher Characteristics, Impact Sample 


Jumpstart CCT-Only Business as 
Characteristic (%) Group Group Usual Group 
Age 
Under 25 0.0 0.0 0.0 
25-29 0.0 0.0 0.0 
30-39 33.3 20.0 16.7 
40-49 16.7 40.0 0.0 
50-59 50.0 40.0 66.7 
60 or more 0.0 0.0 16.7 
Hispanic, Latino, or Spanish 50.0 50.0 50.0 
Race 
Asian 0.0 0.0 25.0 
Black or African American 60.0 40.0 50.0 
Native American 0.0 0.0 0.0 
Pacific Islander 0.0 0.0 0.0 
White 60.0 0.0 25.0 
Other 0.0 60.0 * 0.0 
Highest Education Level 
Some high school 0.0 0.0 0.0 
High school or GED 0.0 0.0 0.0 
Child Development Associate 0.0 0.0 0.0 
Some college 16.7 0.0 0.0 
Associate's degree 0.0 0.0 16.7 
Bachelor's degree 50.0 100.0 83.3 
Graduate degree 16.7 0.0 0.0 
Other 16.7 0.0 0.0 
Years as an early childhood educator 16.7 17.8 24.0 
Years at current early childhood center 8.7 10.8 12.5 
Sample size 6 5 6 


SOURCE: MDRC calculations based on teacher surveys distributed and collected in spring 2018. 


NOTES: Findings in this table are based on data for 6 Jumpstart group teachers, 5 CCT-only group 
teachers, and 6 business as usual group teachers. Sample size for individual items varies due to 
nonresponse or skipped questions. Responses may add up to more that 100 percent as teachers 
"checked all that applied" for certain questions. 

A t-test was used to assess the statistical significance of each estimated difference with significance 
levels as follows: ***=1 percent, **=5 percent, and *=10 percent. Both Jumpstart and CCT-only groups 
were compared to the business as usual baseline. 
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Appendix G 


Statistical Analysis of Impacts 


The impact of the full Jumpstart model was estimated by fitting the following statistical model to children 
in the analysis sample who were assigned to the Jumpstart classrooms or the business as usual classrooms: 


Y; = Q + BoFULL; + x7 Ba + >, AXe +e; 
a s 


where the variables in this model are defined as follows: 


Y; = Outcome of interest for child i; 


ty 

cS 

é 

ae 
| 


Indicator for group membership (=1 if child i is assigned to the 
Jumpstart classroom in their center, and =0 if the child is assigned 
to a business as usual classroom); 


Baj 7 Set of D indicators for the random assignment blocks (=1 if child 
iis in random assignment block d, and =0 otherwise). As noted 
earlier, random assignment was conducted by center, and in some 
centers also by gender and/or by age. 


Xi = A set of Schild-level covariates for child i; these variables are in- 
cluded to improve the precision of the impact estimates; 


Ej = A child-level random error for child i, assumed to be inde- 
pendently and identically distributed. 


In this model, the estimate of fo is a fixed-effects estimate of the impact of the full Jumpstart model in the 
centers and classrooms in the study.' There are several features to note about this model: 


e Random assignment blocks: The model includes a set of indicators for random assign- 
ment blocks. Controlling for blocks in the model serves two purposes. First, it accounts 
for differences in the random assignment ratio (proportion of children assigned to each 
experimental group) across blocks to provide an unbiased estimate of impacts.” Second, 
it improves the precision of the impact estimates by explaining some of the variation in 
child outcomes among children in the sample. 


e Child-level covariates: The model also controls for children’s baseline characteristics, 
including their age, gender, whether the child is a dual language learner, whether a child 
has an individualized education plan, language spoken at home, parent education, family 
size, whether a child’s family received food stamps, the number of times the family has 
moved in the last five years, and children’s baseline (fall 2017) language and literacy 
scores and social-emotional development scores on the DRDP-PS. Controlling for 


‘Tn other words, the results only apply to the centers and classrooms in the study, and they should not be generalized to 
a different set of centers or to a different set of classrooms within the existing centers. 

There are several ways to account for variation in the random assignment ratio. The two most common are to: (1) 
“block-mean” center the covariates on the right-hand side of the model, or (2) include block indicators in the model. These 
two methods produce the same impact estimate. See Raudenbush (2009). 
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children’s baseline characteristics and outcomes improves the precision of the estimated 
impact. 


A similar model was used to estimate the classroom-level impact of CCT, except that in this case, 
the model was fitted to the sample of children in CCT-only and business as usual classes, and the treatment 
variable was an indicator for whether children were assigned to the CCT-only class or the business as usual 
class.? 


3Rather than estimating two separate models, another strategy would have been to estimate the effect of the full 
Jumpstart model and of CCT with the same impact model, by using the full analysis sample and including two treatment 
indicators in the model (one for children in the Jumpstart classroom, and one for children in the CCT-only classroom). The 
two approaches produce very similar results. 
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